Using Vectors with Apache Cassandra and DataStax Astra DB (part 3)

Welcome to half three of my sequence on learnings from constructing a vector database. Partially one, we lined a number of the fundamentals about vectors. Half two was a deep dive into completely different similarities, on and off the unit sphere. Right here, we’ll discover using vectors with Apache Cassandra and DataStax Astra DB.

Similarity search and approximate nearest neighbor

The important thing interrogation one needs to run in a vector database is “similarity search”: given a sure question vector V, one needs the entries whose vectors have the best similarity to V. The subsequent drawback is, methods to make vector search environment friendly when there are probably many thousands and thousands of vectors within the database. Fortunately, there are a number of superior algorithms to do this inside a DB engine, normally involving the so-called Approximate Nearest Neighbour (ANN) search someway. In observe which means one trades the (very small) probability of lacking some matches for a (substantial) acquire in efficiency.

I will not spend many phrases right here on the subject of methods to equip Cassandra with state-of-the-art ANN search: in case you are curious, take a look at this excellent article that tells the story of how we did it! I am going to simply point out that, by sustaining a vector-specialized index alongside the desk, Cassandra / Astra DB can provide versatile, performant and massively scalable vector search. Crucially, this index (a sure sort of SAI index, in Cassandra parlance) wants a selected selection of measure proper from the beginning.

I am going to now take a more in-depth have a look at how the interplay with Cassandra – or equivalently Astra DB – works. You will notice vector-search queries expressed within the Cassandra Query Language (CQL). CQL is a robust method to work together with the database, and vector-related workloads aren’t any exception.

This part assumes CQL is used all through to work together with the database (the reason being that CQL is permissive sufficient to allow you to violate the querying greatest practices as outlined beneath).

Now, I personally suppose CQL is nice, however I respect that not everybody might need the time, the motivation, or the necessity to learn to use it.

Because of this, there are a number of higher-level abstractions that successfully “disguise” the CQL foundations and permit one to concentrate on the vector-related software itself. I am going to simply point out, for Python customers, CassIO, which in flip powers LangChain’s and LlamaIndex’s plugins for Cassandra. For an easier-to-use and cross-language expertise, check out the REST-based Data API accessible in Astra DB.

Index time versus question time

The 2 primary operations when utilizing a vector-enabled database desk are: inserting entries, and working ANN searches. Truly, there’s one other necessary ingredient that comes earlier than every other: creating the desk, together with the required SAI index for vector search. Right here is how the everyday utilization may unfold:

• Creation of desk and index That is the place you select a measure for the desk (e.g. Euclidean).
• Writing entries to the desk (i.e. rows with their vector). All vectors will need to have the dimensionality specified when creating the desk.
• Querying with ANN search “Give me the okay entries whose vector is probably the most much like a question vector V“.
• Additional writing and querying in any order (and many others, and many others …)

The straightforward sequence above stresses a key reality: you decide to a measure proper once you create a desk. Except you drop the index and re-create it anew, any ANN search you will run will use that measure. In different phrases, the measure used for computing similarities is mounted at index-time.

Right here is the CQL code for a minimal creation of a vector desk with the associated index:

CREATE TABLE my_v_table
  (id TEXT PRIMARY KEY, my_vector VECTOR<FLOAT, 3>);
CREATE CUSTOM INDEX my_v_index ON my_v_table(my_vector)
  USING 'StorageAttachedIndex'
  WITH OPTIONS = {'similarity_function': 'EUCLIDEAN'};

For the sake of completeness, right here is the CQL to insert a row within the desk:

INSERT INTO my_v_table
  (id, my_vector)
VALUES
  ('mill_falcon', [8, 3.5, -4.2]);

Lastly, that is how an ANN search is expressed:

SELECT id, my_vector
  FROM my_v_table
  ORDER BY my_vector ANN OF [5.0, -1.0, 6.5]
  LIMIT 5;

As you see, no measure is specified at query-time anymore. The CQL for vector search admits different bits and choices I will not point out right here (be happy to go to the docs page), however there is a vital factor you possibly can add. Observe:

SELECT
  id,
  my_vector,
  similarity_euclidean(my_vector, [5.0, -1.0, 6.5]) as sim
FROM my_v_table
  ORDER BY my_vector ANN OF [5.0, -1.0, 6.5]
  LIMIT 5;

This final assertion could be phrased as:

Give me the 5 most-similar rows to a question vector V and, for every of them, inform me additionally the euclidean similarity between the row and V itself.

The brand new bit is that one asks for a further sim column (the alias is strongly instructed right here, to keep away from unwieldy column names) containing the numeric worth of the similarity between the row and the supplied vector. This similarity is just not saved anyplace on the DB — and the way might it’s, as its worth relies on the vector supplied within the question itself?

Now concentrate: nothing prevents one from utilizing two completely different vectors within the question, and even to ask for a similarity computed with a unique measure than the one chosen earlier for the index (therefore used for looking out)!

// Warning: WEIRD QUERY
//   (verify the vector values and the chosen measure).
// Why do you have to do that?
SELECT
  id,
  my_vector,
  similarity_dot_product(my_vector, [-7, 0, 11]) as weird_sim
FROM my_v_table
  ORDER BY my_vector ANN OF [5.0, -1.0, 6.5]
  LIMIT 5;

In phrases: give me the 5 rows closest to V, in keeping with the Euclidean similarity, and compute their Dot-product similarity to this different vector W.

Fairly odd, no? The purpose is, that is one thing CQL doesn’t actively stop, however that doesn’t imply you ought to do it:

• Utilizing two completely different similarities at index-time and query-time might be by no means a good choice.
• Utilizing two completely different vectors for the ANN half and the sim column (i.e. W != V) is one thing that hardly is smart; specifically, the returned rows wouldn’t be sorted highest-to-lowest-similarity anymore.

In brief, the measure for the sim column is specified at query-time. Furthermore, the question vector and the one for the sim column are handed as two separate parameters.

The sensible recommendation is then this: don’t move two completely different vectors, and don’t combine similarities between index and question.

As normal, the confusion from mixing similarities is tremendously diminished if engaged on the sphere (the ordering of outcomes, not less than, wouldn’t “look bizarre”).

Now, there may be very particular use instances that might knowingly make the most of this type of CQL freedom. However, frankly, in case you are into that form of factor, you already know the subject lined on this part (whilst you’re at it, why do not you drop a remark beneath in your use case? I might be curious to listen to about it).

Be aware: when utilizing the Knowledge API, versus plain CQL, you don’t retain the liberty to ask for a similarity apart from the one configured for the desk index, in addition to the liberty to make use of W != V in vector searches. And, as was argued on this part, this isn’t a nasty factor in any respect!

Caption: Unhealthy (or not less than questionable) practices with vector searches in CQL. Left: SELECT similarity_euclidean(my_vector, V) as sim ... ORDER BY my_vector ANN OF V on a desk created with the Cosine similarity. The returned rows will likely be r1 and r2 in that order, however since (Euclidean!) δ1 > δ2, then the sim column will come again in growing order. Proper: SELECT similarity_euclidean(my_vector, W) as sim ... ORDER BY my_vector ANN OF V on a desk created with Euclidean. The closest-to-farthest sorting of the rows is referred to V, however the similarity is calculated on W: so, one will get again rows r1 and r2 in that order, however the sim column lists growing values once more.

Null vectors and cosine

A detailed have a look at how the similarities are computed will reveal a property particular to the cosine selection: for the null vector, i.e. a vector with all parts equal to zero, the definition is senseless in any respect!

Beneath the “technical” drawback of a division by zero (|v|=0) arising within the similarity system, certainly, lies the deeper mathematical purpose: a section with zero size has no outlined route to talk of.

What does this indicate? In observe, when you plan to make use of the cosine similarity, you will need to be sure that solely non-null vectors are inserted. However then once more, when you select this similarity, you may as properly rescale every incoming vector to unit size and dwell within the consolation of the sphere … which is one thing that may be completed on all vectors besides the null vector! So I’ve come full circle.

Be aware that the null vector has no drawback by any means with the Euclidean distance, nor with the Dot-product (however the latter, as remarked earlier, most likely makes not a lot sense out of the sphere).

So, by now you should be curious as to how Cassandra / Astra DB handles this case. Listed below are the assorted instances:

• Operating a search corresponding to SELECT ... ANN OF [0, 0, ...] raises a database error corresponding to Operation failed – acquired 0 responses and a couple of failures: UNKNOWN from …
• Inserting a row, e.g. INSERT INTO ... VALUES (..., [0, 0, ...], ...) raises the identical error from the DB: Operation failed – acquired 0 responses and a couple of failures: UNKNOWN from …
• Utilizing the null vector for a similarity column computed throughout the question, corresponding to SELECT ... similarity_cosine(my_vector, [0, 0, ...]) ..., returns a sim column of all NaN (not-a-number) values. This stays true additionally when working queries via the Python Cassandra driver.
• Probably the most attention-grabbing case is when the desk already comprises some null-vector rows earlier than the vector SAI index is created. In that case, no errors are raised, however the rows will likely be silently unnoticed of the index and by no means reached by ANN searches. This may be tough, and needs to be stored in thoughts.

Be aware: when utilizing the Knowledge API, the underlying conduct for Cosine-based collections is similar as outlined above – besides, the final case is just not actually doable because the index is created as quickly as the gathering is obtainable.

The primary three installments of this mini-series cowl most of what you will have to know your approach across the arithmetic, and the sensible elements, of similarity computations between vectors. Nonetheless, it seems that just a few obstacles might get in your approach if you wish to carry out one thing as mundane as, say, switching between vector shops in an current GenAI software. Within the subsequent, and final, article, we’ll have a look at a whole instance of such a migration, paying shut consideration to the hidden surprises alongside the best way. See you subsequent time!