IVF-PQ

UDTF: cuvs_ivf_pq

Official cuVS reference: C API

Query-local product-quantized inverted-file nearest-neighbor search.

Quickstart

The call below expects the registered relations dataset_vectors (the dataset role) and query_vectors (the queries role), each passed as a parenthesized SELECT subquery. Substitute your own relations and column names.

SELECT *
FROM cuvs_ivf_pq(
  dataset => (SELECT id, d0, d1 FROM dataset_vectors),
  queries => (SELECT id, d0, d1 FROM query_vectors),
  k => 8,
  metric => 'l2_expanded',
  n_lists => 16,
  n_probes => 16,
  pq_dim => 2
)
ORDER BY query_ordinal, rank;

Inputs

Each relation argument is a parenthesized SELECT subquery that the planner keeps as a real child; metadata validation resolves a registered table or view for the same role instead. See Vector Inputs for the relation identity rules and the ID, dense-vector type, null, finite-value, and runtime-dimension contract.

Role	Required	Validation reference	Description
dataset	yes	table	Dense-vector rows indexed for nearest-neighbor search.
queries	yes	table	Dense-vector query rows matched against the evaluated dataset.

Vector element types

Element type	Valid metrics
Float32	l2_expanded, l2_sqrt_expanded, cosine, inner_product
Int8	l2_expanded, l2_sqrt_expanded, inner_product
UInt8	l2_expanded, l2_sqrt_expanded, inner_product

Arguments and options

Scalar SQL arguments

Argument	Type	Required	Description
k	integer	yes	Number of neighbors returned for each evaluated query row.
metric	enum ("l2_expanded", "l2_sqrt_expanded", "cosine", "inner_product")	no	Distance or score metric; native default l2_expanded. Distance metrics rank lower values first; inner product ranks higher values first.
n_lists	integer	yes	Number of inverted lists; must not exceed the evaluated dataset row count.
n_probes	integer	no	Lists probed per query; native default 20, and must not exceed n_lists.
kmeans_n_iters	integer	no	Iterations for training list centers; native default 20.
kmeans_trainset_fraction	number	no	Fraction of the dataset used for training centers; native default 0.5.
pq_dim	integer	no	Compressed dimensions; native default 0 selects the dimension automatically. pq_dim * pq_bits must be divisible by 8.
pq_bits	integer	no	Bits per PQ code; native default 8.
codebook_kind	enum ("per_subspace", "per_cluster")	no	PQ codebook training scope; native default per_subspace.

SQL value argument schemas

Argument	Required	Literal shape	Default	Constraints	Description
codebook_kind	no	string	"per_subspace"	one of "per_subspace", "per_cluster"	PQ codebook training scope; native default per_subspace.
k	yes	integer	No default	minimum 1; maximum 4294967295	Number of neighbors returned for each evaluated query row.
kmeans_n_iters	no	integer	20	minimum 1; maximum 4294967295	Iterations for training list centers; native default 20.
kmeans_trainset_fraction	no	number	0.5	greater than 0; maximum 1	Fraction of the dataset used for training centers; native default 0.5.
metric	no	string	"l2_expanded"	one of "l2_expanded", "l2_sqrt_expanded", "cosine", "inner_product"	Distance or score metric; native default l2_expanded. Distance metrics rank lower values first; inner product ranks higher values first. Supported element/metric combinations: Float32: l2_expanded, l2_sqrt_expanded, cosine, inner_product; Int8: l2_expanded, l2_sqrt_expanded, inner_product; UInt8: l2_expanded, l2_sqrt_expanded, inner_product.
n_lists	yes	integer	No default	minimum 1; maximum 4294967295	Number of inverted lists; must not exceed the evaluated dataset row count.
n_probes	no	integer	20	minimum 1; maximum 4294967295	Lists probed per query; native default 20, and must not exceed n_lists.
pq_bits	no	integer	8	minimum 4; maximum 8	Bits per PQ code; native default 8.
pq_dim	no	integer	0	minimum 0; maximum 4294967295	Compressed dimensions; native default 0 selects the dimension automatically. pq_dim * pq_bits must be divisible by 8.

Vector binding shapes

Each relation subquery must project a non-null id field followed by either one or more non-null feature fields of a supported element type (Float32, Int8, UInt8) or one non-null list vector field named vector.

For wide vectors, the projection order defines the feature dimensions. A list vector relation must contain no feature field beside id and vector.

Output

Column	Type	Nullable	Description
query_ordinal	UInt64	no	Zero-based ordinal of the evaluated query row; it disambiguates duplicate query IDs.
query_id	same_as_queries.id	no	Logical ID copied from the queries relation.
neighbor_ordinal	UInt64	no	Zero-based ordinal of the matched dataset row; it disambiguates duplicate dataset IDs.
neighbor_id	same_as_dataset.id	no	Logical ID copied from the matched dataset row.
rank	UInt32	no	One-based neighbor rank within a query. Order consumers explicitly by query_ordinal, rank.
distance	Float32	no	Metric value; smaller is better for distance metrics, while inner_product prefers larger values.

Concrete schemas are call-specific. Run gpu_validate_call against registered relations to inspect the output schema after the actual ID types and literal options are validated.

Limits

• Validation resolves named tables or views and reads schemas only; it does not execute relation scans or GPU work.
• Execution relation arguments require parenthesized subqueries; dry-run validation accepts registered named relations only.
• Builds and destroys its query-local index within the statement; no index persists across statements.
• Product quantization is lossy even when all lists are probed. Missing neighbors are omitted and valid ranks remain contiguous.
• The dataset must be non-empty; k and n_lists must not exceed its row count, and n_probes must not exceed n_lists.
• Dataset and query dimensions and element types must match. Cosine requires at least two dimensions. Nonzero pq_dim * pq_bits must be divisible by 8; dimensions are padded when needed.

To dry-run validate relation metadata, column types, and options without execution, see gpu_validate_call.