CAGRA

UDTF: cuvs_cagra

Official cuVS reference: C API

Query-local graph approximate nearest-neighbor search.

Quickstart

The call below expects the registered relations dataset_vectors (the dataset role) and query_vectors (the queries role), each passed as a parenthesized SELECT subquery.

SELECT *
FROM cuvs_cagra(
  dataset => (SELECT id, d0, d1 FROM dataset_vectors),
  queries => (SELECT id, d0, d1 FROM query_vectors),
  k => 8,
  metric => 'l2_expanded',
  graph_degree => 32,
  intermediate_graph_degree => 64
)
ORDER BY query_ordinal, rank;

Inputs

Each relation argument is a parenthesized SELECT subquery that the planner keeps as an operator child. Metadata validation resolves a registered table or view for the same role. See Vector Inputs for the relation identity rules and the ID, dense-vector type, null, finite-value, and runtime-dimension contract.

Role	Required	Validation reference	Description
dataset	yes	table	Dense-vector rows indexed for nearest-neighbor search.
queries	yes	table	Dense-vector query rows matched against the evaluated dataset.

Vector element types

Element type	Valid metrics
Float32	l2_expanded, inner_product, cosine
Int8	l2_expanded, inner_product, cosine
UInt8	l2_expanded, inner_product, cosine

Arguments and options

Scalar SQL arguments

Argument	Type	Required	Default	Constraints	Description
build_algo	string	no	"ivf_pq"	one of "ivf_pq", "nn_descent"	Graph construction algorithm; native default ivf_pq. nn_descent does not support cosine.
graph_degree	integer	no	64	minimum 1; maximum 4294967295	Output graph degree; native default 64. Must be less than dataset rows.
intermediate_graph_degree	integer	no	128	minimum 1; maximum 4294967295	Build graph degree; native default 128. Must exceed graph_degree and be less than dataset rows.
itopk_size	integer	no	64	minimum 1; maximum 4294967295	Intermediate search results; native default 64. Must be at least k.
k	integer	yes	No default	minimum 1; maximum 4294967295	Number of neighbors returned for each evaluated query row.
max_iterations	integer	no	0	minimum 0; maximum 4294967295	Maximum search iterations; native default 0 selects automatically.
metric	string	no	"l2_expanded"	one of "l2_expanded", "inner_product", "cosine"	Distance or score metric; native default l2_expanded. inner_product ranks higher scores first.
search_algo	string	no	"auto"	one of "auto", "single_cta", "multi_cta", "multi_kernel"	Search kernel; auto is set explicitly (the native zero-initialized default is single_cta).
search_width	integer	no	1	minimum 1; maximum 4294967295	Starting graph nodes per iteration; native default 1.

Vector binding shapes

Each relation subquery must project an id field followed by either one or more feature fields of a supported element type (Float32, Int8, UInt8) or one list field of a supported element type, under any name.

For wide vectors, the projection order defines the feature dimensions. A list vector relation must contain no feature field beside id and its list field. Nullable field types are accepted. A null ID, feature value, list row, or list element fails the query.

Output

Column	Type	Nullable	Description
query_ordinal	UInt64	no	Zero-based ordinal of the evaluated query row; it disambiguates duplicate query IDs.
query_id	same_as_queries.id	no	Logical ID copied from the queries relation.
neighbor_ordinal	UInt64	no	Zero-based ordinal of the matched dataset row; it disambiguates duplicate dataset IDs.
neighbor_id	same_as_dataset.id	no	Logical ID copied from the matched dataset row.
rank	UInt32	no	One-based neighbor rank within a query. Order consumers explicitly by query_ordinal, rank.
distance	Float32	no	Metric value; smaller is better for distance metrics, while inner_product prefers larger values.

The output schema depends on the call. Run gpu_validate_call against registered relations to inspect the schema that the validated ID types and literal options produce.

Examples

MerRec

Which neighbor matches survive approximate search?

The MerRec dataset supplies mr_taste64, a prepared table of category-view-share vectors for 2,080,743 shoppers with at least 20 views before September 1. Each method searches the table against itself and returns ten neighbors per shopper.

Recall compares neighbor IDs with exact search. Some shoppers have identical vectors, so the last column also checks whether a returned distance is within 1e-5 of the exact tenth-neighbor distance.

Method	Exact neighbor IDs kept	Within exact 10th distance
Exact brute force	100%	100%
CAGRA	99.3%	99.9%
IVF-Flat	98.2%	99.0%
IVF-PQ	86.1%	90.4%

CAGRA and IVF-Flat retain more than 98% of exact neighbor IDs for this batch, while IVF-PQ retains 86.1%. These methods build a new index on every run, so a smaller query batch still pays that construction cost. Choosing a method depends on both recall and the expected batch size.

Executed SQL: CAGRA neighbors and recall

SELECT COUNT(*) AS neighbor_rows, COUNT(DISTINCT query_id) AS shoppers, AVG(distance) AS mean_distance
FROM cuvs_cagra(
  dataset => (SELECT * FROM mr_taste64 ORDER BY id),
  queries => (SELECT * FROM mr_taste64 ORDER BY id),
  k => 10, graph_degree => 32, intermediate_graph_degree => 64, build_algo => 'nn_descent'
);

WITH exact AS (
  SELECT query_id, neighbor_id, distance FROM cuvs_brute_force_knn(
    dataset => (SELECT * FROM mr_taste64 ORDER BY id),
    queries => (SELECT * FROM mr_taste64 ORDER BY id),
    k => 10, metric => 'l2_expanded')
), approx AS (
  SELECT query_id, neighbor_id, distance FROM cuvs_cagra(
    dataset => (SELECT * FROM mr_taste64 ORDER BY id),
    queries => (SELECT * FROM mr_taste64 ORDER BY id),
    k => 10, graph_degree => 32, intermediate_graph_degree => 64, build_algo => 'nn_descent')
), kth AS (
  SELECT query_id, MAX(distance) AS kth_distance FROM exact GROUP BY query_id
)
SELECT COUNT(*) AS approx_rows,
       CAST(COUNT(e.neighbor_id) AS DOUBLE) / COUNT(*) AS id_recall,
       AVG(CASE WHEN a.distance <= k.kth_distance + 1e-5 THEN 1.0 ELSE 0.0 END) AS within_exact_kth
FROM approx a
JOIN kth k ON k.query_id = a.query_id
LEFT JOIN exact e ON e.query_id = a.query_id AND e.neighbor_id = a.neighbor_id;

Executed SQL: IVF-Flat neighbors and recall

SELECT COUNT(*) AS neighbor_rows, COUNT(DISTINCT query_id) AS shoppers, AVG(distance) AS mean_distance
FROM cuvs_ivf_flat(
  dataset => (SELECT * FROM mr_taste64 ORDER BY id),
  queries => (SELECT * FROM mr_taste64 ORDER BY id),
  k => 10, n_lists => 2048, n_probes => 32
);

WITH exact AS (
  SELECT query_id, neighbor_id, distance FROM cuvs_brute_force_knn(
    dataset => (SELECT * FROM mr_taste64 ORDER BY id),
    queries => (SELECT * FROM mr_taste64 ORDER BY id),
    k => 10, metric => 'l2_expanded')
), approx AS (
  SELECT query_id, neighbor_id, distance FROM cuvs_ivf_flat(
    dataset => (SELECT * FROM mr_taste64 ORDER BY id),
    queries => (SELECT * FROM mr_taste64 ORDER BY id),
    k => 10, n_lists => 2048, n_probes => 32)
), kth AS (
  SELECT query_id, MAX(distance) AS kth_distance FROM exact GROUP BY query_id
)
SELECT COUNT(*) AS approx_rows,
       CAST(COUNT(e.neighbor_id) AS DOUBLE) / COUNT(*) AS id_recall,
       AVG(CASE WHEN a.distance <= k.kth_distance + 1e-5 THEN 1.0 ELSE 0.0 END) AS within_exact_kth
FROM approx a
JOIN kth k ON k.query_id = a.query_id
LEFT JOIN exact e ON e.query_id = a.query_id AND e.neighbor_id = a.neighbor_id;

Executed SQL: IVF-PQ neighbors and recall

SELECT COUNT(*) AS neighbor_rows, COUNT(DISTINCT query_id) AS shoppers, AVG(distance) AS mean_distance
FROM cuvs_ivf_pq(
  dataset => (SELECT * FROM mr_taste64 ORDER BY id),
  queries => (SELECT * FROM mr_taste64 ORDER BY id),
  k => 10, n_lists => 2048, n_probes => 64, pq_dim => 32, pq_bits => 8
);

WITH exact AS (
  SELECT query_id, neighbor_id, distance FROM cuvs_brute_force_knn(
    dataset => (SELECT * FROM mr_taste64 ORDER BY id),
    queries => (SELECT * FROM mr_taste64 ORDER BY id),
    k => 10, metric => 'l2_expanded')
), approx AS (
  SELECT query_id, neighbor_id, distance FROM cuvs_ivf_pq(
    dataset => (SELECT * FROM mr_taste64 ORDER BY id),
    queries => (SELECT * FROM mr_taste64 ORDER BY id),
    k => 10, n_lists => 2048, n_probes => 64, pq_dim => 32, pq_bits => 8)
), kth AS (
  SELECT query_id, MAX(distance) AS kth_distance FROM exact GROUP BY query_id
)
SELECT COUNT(*) AS approx_rows,
       CAST(COUNT(e.neighbor_id) AS DOUBLE) / COUNT(*) AS id_recall,
       AVG(CASE WHEN a.distance <= k.kth_distance + 1e-5 THEN 1.0 ELSE 0.0 END) AS within_exact_kth
FROM approx a
JOIN kth k ON k.query_id = a.query_id
LEFT JOIN exact e ON e.query_id = a.query_id AND e.neighbor_id = a.neighbor_id;

Executed SQL: exact baseline and mr_taste64 preparation

SELECT COUNT(*) AS neighbor_rows, COUNT(DISTINCT query_id) AS shoppers, AVG(distance) AS mean_distance
FROM cuvs_brute_force_knn(
  dataset => (SELECT * FROM mr_taste64 ORDER BY id),
  queries => (SELECT * FROM mr_taste64 ORDER BY id),
  k => 10, metric => 'l2_expanded'
);

WITH views AS (
  SELECT e.user_id, i.c1_id, i.brand_id
  FROM merrec_events e JOIN merrec_items i ON i.item_id = e.item_id
  WHERE e.event_type = 'item_view' AND e.event_time < TIMESTAMP '2023-09-01 00:00:00'
), dims AS (
  SELECT c1_id, ROW_NUMBER() OVER (ORDER BY n DESC, c1_id) - 1 AS dim
  FROM (SELECT c1_id, COUNT(*) AS n FROM views GROUP BY c1_id) c
  ORDER BY n DESC LIMIT 64
), counts AS (
  SELECT v.user_id, d.dim, COUNT(*) AS n FROM views v JOIN dims d ON d.c1_id = v.c1_id GROUP BY v.user_id, d.dim
), totals AS (
  SELECT user_id, SUM(n) AS total FROM counts GROUP BY user_id HAVING SUM(n) >= 20
), taste AS (
  SELECT c.user_id AS id,
         CAST(SQRT(SUM(CASE WHEN c.dim = 0 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d0,
         CAST(SQRT(SUM(CASE WHEN c.dim = 1 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d1,
         CAST(SQRT(SUM(CASE WHEN c.dim = 2 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d2,
         CAST(SQRT(SUM(CASE WHEN c.dim = 3 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d3,
         CAST(SQRT(SUM(CASE WHEN c.dim = 4 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d4,
         CAST(SQRT(SUM(CASE WHEN c.dim = 5 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d5,
         CAST(SQRT(SUM(CASE WHEN c.dim = 6 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d6,
         CAST(SQRT(SUM(CASE WHEN c.dim = 7 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d7,
         CAST(SQRT(SUM(CASE WHEN c.dim = 8 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d8,
         CAST(SQRT(SUM(CASE WHEN c.dim = 9 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d9,
         CAST(SQRT(SUM(CASE WHEN c.dim = 10 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d10,
         CAST(SQRT(SUM(CASE WHEN c.dim = 11 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d11,
         CAST(SQRT(SUM(CASE WHEN c.dim = 12 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d12,
         CAST(SQRT(SUM(CASE WHEN c.dim = 13 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d13,
         CAST(SQRT(SUM(CASE WHEN c.dim = 14 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d14,
         CAST(SQRT(SUM(CASE WHEN c.dim = 15 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d15,
         CAST(SQRT(SUM(CASE WHEN c.dim = 16 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d16,
         CAST(SQRT(SUM(CASE WHEN c.dim = 17 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d17,
         CAST(SQRT(SUM(CASE WHEN c.dim = 18 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d18,
         CAST(SQRT(SUM(CASE WHEN c.dim = 19 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d19,
         CAST(SQRT(SUM(CASE WHEN c.dim = 20 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d20,
         CAST(SQRT(SUM(CASE WHEN c.dim = 21 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d21,
         CAST(SQRT(SUM(CASE WHEN c.dim = 22 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d22,
         CAST(SQRT(SUM(CASE WHEN c.dim = 23 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d23,
         CAST(SQRT(SUM(CASE WHEN c.dim = 24 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d24,
         CAST(SQRT(SUM(CASE WHEN c.dim = 25 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d25,
         CAST(SQRT(SUM(CASE WHEN c.dim = 26 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d26,
         CAST(SQRT(SUM(CASE WHEN c.dim = 27 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d27,
         CAST(SQRT(SUM(CASE WHEN c.dim = 28 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d28,
         CAST(SQRT(SUM(CASE WHEN c.dim = 29 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d29,
         CAST(SQRT(SUM(CASE WHEN c.dim = 30 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d30,
         CAST(SQRT(SUM(CASE WHEN c.dim = 31 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d31,
         CAST(SQRT(SUM(CASE WHEN c.dim = 32 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d32,
         CAST(SQRT(SUM(CASE WHEN c.dim = 33 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d33,
         CAST(SQRT(SUM(CASE WHEN c.dim = 34 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d34,
         CAST(SQRT(SUM(CASE WHEN c.dim = 35 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d35,
         CAST(SQRT(SUM(CASE WHEN c.dim = 36 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d36,
         CAST(SQRT(SUM(CASE WHEN c.dim = 37 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d37,
         CAST(SQRT(SUM(CASE WHEN c.dim = 38 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d38,
         CAST(SQRT(SUM(CASE WHEN c.dim = 39 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d39,
         CAST(SQRT(SUM(CASE WHEN c.dim = 40 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d40,
         CAST(SQRT(SUM(CASE WHEN c.dim = 41 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d41,
         CAST(SQRT(SUM(CASE WHEN c.dim = 42 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d42,
         CAST(SQRT(SUM(CASE WHEN c.dim = 43 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d43,
         CAST(SQRT(SUM(CASE WHEN c.dim = 44 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d44,
         CAST(SQRT(SUM(CASE WHEN c.dim = 45 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d45,
         CAST(SQRT(SUM(CASE WHEN c.dim = 46 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d46,
         CAST(SQRT(SUM(CASE WHEN c.dim = 47 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d47,
         CAST(SQRT(SUM(CASE WHEN c.dim = 48 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d48,
         CAST(SQRT(SUM(CASE WHEN c.dim = 49 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d49,
         CAST(SQRT(SUM(CASE WHEN c.dim = 50 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d50,
         CAST(SQRT(SUM(CASE WHEN c.dim = 51 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d51,
         CAST(SQRT(SUM(CASE WHEN c.dim = 52 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d52,
         CAST(SQRT(SUM(CASE WHEN c.dim = 53 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d53,
         CAST(SQRT(SUM(CASE WHEN c.dim = 54 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d54,
         CAST(SQRT(SUM(CASE WHEN c.dim = 55 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d55,
         CAST(SQRT(SUM(CASE WHEN c.dim = 56 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d56,
         CAST(SQRT(SUM(CASE WHEN c.dim = 57 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d57,
         CAST(SQRT(SUM(CASE WHEN c.dim = 58 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d58,
         CAST(SQRT(SUM(CASE WHEN c.dim = 59 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d59,
         CAST(SQRT(SUM(CASE WHEN c.dim = 60 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d60,
         CAST(SQRT(SUM(CASE WHEN c.dim = 61 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d61,
         CAST(SQRT(SUM(CASE WHEN c.dim = 62 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d62,
         CAST(SQRT(SUM(CASE WHEN c.dim = 63 THEN c.n ELSE 0 END) * 1.0 / t.total) AS REAL) AS d63
  FROM counts c JOIN totals t ON t.user_id = c.user_id
  GROUP BY c.user_id, t.total
)
SELECT * FROM taste
ORDER BY id;

EMBER2024

Which earlier files resemble a new-quarter file?

This EMBER2024 example searches earlier Win32 files for structural references to files in a later test period. The current relations contain the full deduplicated splits. The corpus has 1,560,000 Win32 training files from weeks 0 to 51; the query selects the first 10,000 Win32 test files by ID from weeks 52 to 63. Each file is described by a 512-element normalized byte and byte-entropy histogram.

The exact search returns ten historical references for each test query, 100,000 rows in all. It passes only IDs and vectors to cuvs_brute_force_knn. Labels, family names, weeks and SHA-256 values join after the search. For test file 10021000 from week 52, the first three references have the saved family field cosmu:

Rank	Historical file	Week	Squared-L2 distance
1	2751446	27	0.00007534
2	3779432	37	0.00009656
3	4070733	40	0.00010586

The shared family field is a reason to inspect these references. It does not make the nearest-neighbor distance a calibrated malware score or classify the query file. The full-corpus capture has no independent CPU recall oracle, so it supports retrieval and row counts without a full-scale quality claim.

Exact search

The exact search returned 100,000 rows.

-- Exact squared-L2 reference search over full Win32 train dataset and test queries.
-- Corpus: full deduplicated Win32 training dataset (~1.56M files) across weeks 0-51.
-- Queries: 10,000 held-out Win32 test queries from weeks 52-63.
-- Post-search enrichment joins query and neighbor metadata only after vector search.
WITH train_index AS (
    SELECT v.id, v.histogram AS vector
    FROM ember_research_vectors_train v
    JOIN ember_metadata_train m ON m.id = v.id
    WHERE m.file_type = 'Win32'
    ORDER BY v.id
), test_queries AS (
    SELECT v.id, v.histogram AS vector
    FROM ember_research_vectors_test v
    JOIN ember_metadata_test m ON m.id = v.id
    WHERE m.file_type = 'Win32'
    ORDER BY v.id
    LIMIT 10000
)
SELECT n.query_ordinal,
       n.query_id,
       qm.week_id AS query_week,
       qm.label AS query_label,
       n.rank,
       n.neighbor_id,
       tm.week_id AS neighbor_week,
       tm.label AS neighbor_label,
       tm.sha256 AS neighbor_sha256,
       tm.family AS neighbor_family,
       n.distance
FROM cuvs_brute_force_knn(
    dataset => (SELECT id, vector FROM train_index ORDER BY id),
    queries => (SELECT id, vector FROM test_queries ORDER BY id),
    k => 10,
    metric => 'l2_expanded'
) n
JOIN ember_metadata_test qm ON qm.id = n.query_id
JOIN ember_metadata_train tm ON tm.id = n.neighbor_id
ORDER BY n.query_ordinal, n.rank;

IVF-Flat

The IVF-Flat query used 1,024 lists and 32 probes over the same full corpus and 10,000-query batch. It returned 100,000 rows. Its first three references for file 10021000 have the same IDs as the exact result. That one example does not establish recall over all queries.

-- IVF-Flat approximate reference search over the same full Win32 train dataset and test queries.
-- Index is built query-locally with 1024 lists and searched with 32 probes.
-- Corpus: full deduplicated Win32 training dataset (~1.56M files).
-- Queries: 10,000 held-out Win32 test queries from weeks 52-63.
WITH train_index AS (
    SELECT v.id, v.histogram AS vector
    FROM ember_research_vectors_train v
    JOIN ember_metadata_train m ON m.id = v.id
    WHERE m.file_type = 'Win32'
    ORDER BY v.id
), test_queries AS (
    SELECT v.id, v.histogram AS vector
    FROM ember_research_vectors_test v
    JOIN ember_metadata_test m ON m.id = v.id
    WHERE m.file_type = 'Win32'
    ORDER BY v.id
    LIMIT 10000
)
SELECT n.query_ordinal,
       n.query_id,
       qm.week_id AS query_week,
       qm.label AS query_label,
       n.rank,
       n.neighbor_id,
       tm.week_id AS neighbor_week,
       tm.label AS neighbor_label,
       tm.sha256 AS neighbor_sha256,
       tm.family AS neighbor_family,
       n.distance
FROM cuvs_ivf_flat(
    dataset => (SELECT id, vector FROM train_index ORDER BY id),
    queries => (SELECT id, vector FROM test_queries ORDER BY id),
    k => 10,
    metric => 'l2_expanded',
    n_lists => 1024,
    n_probes => 32
) n
JOIN ember_metadata_test qm ON qm.id = n.query_id
JOIN ember_metadata_train tm ON tm.id = n.neighbor_id
ORDER BY n.query_ordinal, n.rank;

The full-reference capture records the exact, IVF-Flat and failed CAGRA attempts with their query IDs, source result paths and first-query records. The CAGRA attempt ended during index construction and returned no result rows.

Limits

• Builds and frees its graph and padded dataset during each query execution; executions never share an index.
• Approximate search may return fewer than k neighbors; missing (query, rank) rows are omitted and valid ranks remain contiguous.
• The dataset must be non-empty, k and intermediate_graph_degree must be less than or equal to its row count, and graph_degree must be less than intermediate_graph_degree.
• k must not exceed itopk_size. single_cta requires itopk_size no greater than 1024. cosine cannot use nn_descent.
• Dataset and query dimensions and element types must match.
• Neighbors and distances can differ slightly between runs, because graph construction orders edges by atomic arrival; there is no seed option.