diff --git a/.github/workflows/assemble.yml b/.github/workflows/assemble.yml index c2ce277e..93428a39 100644 --- a/.github/workflows/assemble.yml +++ b/.github/workflows/assemble.yml @@ -43,7 +43,9 @@ jobs: uses: astral-sh/setup-uv@v5 - name: Install Mintlify CLI - run: npm install -g mint + # Keep the two exports byte-comparable. Newer releases can render + # nondeterministic OpenAPI examples from identical input. + run: npm install -g mint@4.2.888 # The OpenAPI spec is tracked at a release tag, not a commit. If the two # have drifted the site would document something that was never released. diff --git a/docs/indexing/vector-index.mdx b/docs/indexing/vector-index.mdx index 0182d04c..80d0e9a5 100644 --- a/docs/indexing/vector-index.mdx +++ b/docs/indexing/vector-index.mdx @@ -135,9 +135,17 @@ Learn how to configure, build, and search LanceDB vector indexes, including buil | `bypass_vector_index()` | Ignore the ANN index entirely and perform an exact (flat) scan. Can be used to measure ANN `recall@k`, or to query with a metric the index was not built for (e.g., a non-cosine query on a multivector column). | -The `distance_type` can be specified at search time in addition to build time. If you do this you must ensure that -you set it to the same value you used when building the vector index. If the search time distance does not match the -distance type set at build time then a costly flat search will be performed. +The `distance_type` can be specified at search time in addition to build time. If +you do this, set it to the same value used to build the vector index. If the search +metric does not match the index metric, LanceDB cannot use the index and falls +back to a costly flat search. On LanceDB Cloud, the brute-force KNN safety limit +can reject that search on a large table instead of executing it. LanceDB +Enterprise doesn't have the limit. + +When the search metric is omitted, current SDKs use the index metric if an index +is available. Some older SDK versions sent `l2` when it was omitted, so explicitly +set the matching metric or upgrade the SDK when querying a non-`l2` index with an +older client. **Recommended `nprobes` behavior by index type:** diff --git a/docs/performance.mdx b/docs/performance.mdx index c7008ed7..7602627f 100644 --- a/docs/performance.mdx +++ b/docs/performance.mdx @@ -124,7 +124,7 @@ If vector search latency climbs with table size (i.e., queries that ran in milli | `IVF_HNSW_SQ` | Best recall/latency for unfiltered search; higher latency variance under selective filters. | | `IVF_FLAT` | Required for binary vectors with `hamming`. | -The distance metric is fixed once the index is built. Pick the distance metric based on how the embedding model was trained: `cosine` (unnormalized), `dot` (already-normalized, best performance), `l2` (general-purpose, default), `hamming` (binary). For parameter tuning, see [Vector Indexing](/indexing/vector-index). +The distance metric is fixed once the index is built. Pick the distance metric based on how the embedding model was trained: `cosine` (unnormalized), `dot` (already-normalized, best performance), `l2` (general-purpose, default), `hamming` (binary). If a query explicitly requests a different metric, the index cannot be used and the query falls back to a brute-force scan; LanceDB Enterprise may reject that scan on a large table. For parameter tuning, see [Vector Indexing](/indexing/vector-index). ### Scalar indexes diff --git a/docs/search/multivector-search.mdx b/docs/search/multivector-search.mdx index 429ce25a..11ed709f 100644 --- a/docs/search/multivector-search.mdx +++ b/docs/search/multivector-search.mdx @@ -131,7 +131,7 @@ tbl.create_index(metric="cosine", vector_column_name="vector") A brute-force scan over a multivector column has to compare every query vector to every document vector in every row, so the cost grows with both the row count and the number of vectors per row. In LanceDB OSS, the query will just run, so a large unindexed multivector table can stall a process for a long time before returning results. -On LanceDB Enterprise, the brute-force KNN safety check applies a stricter row threshold to multivector columns — roughly 10× lower than for single-vector columns. So an unindexed multivector table will start being rejected with a "vector search would use brute-force KNN" error well before a comparable single-vector table would. Build the index before you start hitting it from production traffic, even if the dataset is small enough that you'd skip indexing for a single-vector workload. +On LanceDB Cloud, the brute-force KNN safety check applies a stricter row threshold to multivector columns — roughly 10× lower than for single-vector columns. So an unindexed multivector table can be rejected with a "vector search would use brute-force KNN" error well before a comparable single-vector table would. LanceDB Enterprise doesn't have the limit. Build the index before you start hitting it from production traffic, even if the dataset is small enough that you'd skip indexing for a single-vector workload. ### 6. Query a Single Vector diff --git a/docs/search/vector-search.mdx b/docs/search/vector-search.mdx index a86276f0..84bf2c8d 100644 --- a/docs/search/vector-search.mdx +++ b/docs/search/vector-search.mdx @@ -79,10 +79,18 @@ For indexed search, supported distance metrics vary by index type: ### Configure Distance Metric -By default, `l2` will be used as metric type. You can specify the metric type as -`cosine` or `dot` if required (`hamming` is supported for `IVF_FLAT` index only). +Without a vector index, searches use `l2` by default. You can specify `cosine` or +`dot` instead (`hamming` is supported for `IVF_FLAT` indexes only). -**Note:** You can configure the distance metric during search only if there's no vector index. If a vector index exists, the distance metric will always be the one you specified when creating the index. +When a vector index exists, the search metric must match the metric used to build +the index. If you explicitly request a different metric, LanceDB cannot use that +index and falls back to an exhaustive (brute-force) search. On LanceDB Cloud, the +brute-force KNN safety limit can reject that search on a large table instead of +executing it. LanceDB Enterprise doesn't have the limit. + +If you omit the search metric, current SDKs use the index metric when an index is +available. Some older SDK versions sent `l2` when the metric was omitted; with a +non-`l2` index, explicitly set the matching metric or upgrade the SDK. @@ -98,7 +106,10 @@ By default, `l2` will be used as metric type. You can specify the metric type as -Here you can see the same search but using `cosine` similarity instead of `l2` distance. The result focuses on vector direction rather than absolute distance, which works better for normalized embeddings. +Here you can see the same search but using `cosine` similarity instead of `l2` +distance. If the table has a vector index, make sure it was also built with +`cosine`. The result focuses on vector direction rather than absolute distance, +which works better for unnormalized embeddings. Set `.limit(...)` on vector searches you run in applications. You can page through results with `.offset(...)` and include LanceDB's internal row id with `.with_row_id()` / `.withRowId()` when you need a stable handle for follow-up operations.