TSVD

UDTF: cuml_tsvd

Official cuML reference: Python API

cuML truncated SVD fits and transforms Float32 input rows.

Quickstart

Register the relations referenced by the parenthesized SELECT clauses below. For metadata validation, the descriptor names input_vectors.

SELECT id, pc_0, pc_1
FROM cuml_tsvd(
  input => (SELECT id, d0, d1 FROM t),
  n_components => 2
)
ORDER BY row_ordinal;

Inputs

Each relation argument is a parenthesized SELECT subquery. Metadata validation resolves a registered table or view for the same role without scanning its rows. See ML Inputs for the ID, Float32 feature, null, finite-value, and runtime-dimension contract.

Role	Required	Validation reference	Description
input	yes	table	Dense Float32 rows fitted and transformed in this statement.

Vector element types

Element type	Valid metrics
Float32	Not applicable

Arguments and options

Scalar SQL arguments

Argument	Type	Required	Description
n_components	integer	yes	Number of transformed pc_<index> columns.
solver	enum ("cov_eig_dq", "cov_eig_jacobi")	no	Covariance eigensolver used by the query-local fit.
tol	number	no	Finite nonnegative eigensolver tolerance.
n_iterations	integer	no	Maximum number of eigensolver iterations.

SQL value argument schemas

Argument	Required	Literal shape	Default	Constraints	Description
n_components	yes	integer	No default	minimum 1; maximum 2147483647	Number of transformed pc_<index> columns.
n_iterations	no	integer	15	minimum 1; maximum 4294967295	Maximum number of eigensolver iterations.
solver	no	string	"cov_eig_dq"	one of "cov_eig_dq", "cov_eig_jacobi"	Covariance eigensolver used by the query-local fit.
tol	no	number	0	minimum 0	Finite nonnegative eigensolver tolerance.

Vector binding shapes

Each relation subquery projects a non-null id and either non-null Float32 feature columns in dimension order or one non-null vector list column of non-null Float32 values. For classification, training also projects non-null Int32 label; regression requires non-null finite Float32 label. The predict relation omits it.

The list-column shape excludes other feature columns. See ML Inputs for the allowed list containers and runtime checks.

Output

Column	Type	Nullable	Description
row_ordinal	UInt64	no	Evaluated input row position, including when IDs repeat. Follows the input's top-level ORDER BY on output columns; unspecified without one.
id	same_as_input.id	no	Logical input ID.
pc_<index>	Float32	no	Transformed component value, repeated n_components times. The output contains n_components such columns, pc_0 through pc_&lt;n_components - 1&gt;.

Concrete schemas are call-specific. Run gpu_validate_call against registered relations to inspect the output schema after the actual ID types and literal options are validated.

Examples

This example uses the EMBER2024 demo dataset.

What does two-component truncated SVD show in the challenge cohort?

The query projects all 3,225 Win32 challenge histograms from 512 dimensions to two components. The CPU geometry check measured 62.853% uncentered energy in those components. An independent CPU comparison of the transformed Gram matrix passed with relative Frobenius error 9.668349094726904e-07 against a 0.0001 tolerance.

The saved projections share all 3,225 rows. Switch among their coordinates while filtering by SHA-256, family, or the capture-local cluster IDs assigned by HDBSCAN after PCA-16. Family metadata is joined after the fit and does not enter the TSVD input.

ProjectionUMAPPCATruncated SVDSearch full SHA-256 or familyPCA-16 HDBSCAN clusterAll clustersNoise (-1)Cluster 0Cluster 1Cluster 2Cluster 3Cluster 4Cluster 5Cluster 6

Noise (-1)Cluster 0Cluster 1Cluster 2Cluster 3Cluster 4Cluster 5Cluster 63,225 of 3,225 points match

Focus the plot and use ← / → to navigate, Enter to select.

doenerium

EMBER ID 20000000 · week 0 · capture-local cluster 5 · HDBSCAN membership strength 1.000

058f63432dd8f679aef17b6fa16dd513273f10d8512f9897fa850585ea1d6d02

All 3,225 saved challenge Win32 rows are included. Projection and cluster values come from saved SQL results; changing filters does not rerun SQL. Family is post-fit metadata. HDBSCAN clusters were fitted to PCA-16 coordinates, while this map can display PCA-2, TSVD-2, or UMAP-2. Cluster IDs are local to this capture. Coordinates are visual projections with no calibrated malware-risk interpretation. Noise (-1) records rows left unassigned under these HDBSCAN parameters; that label alone carries no novelty classification. Download the saved capture data.

Saved SQL capture IDs

• PCA-2: 65e2e2b8-9734-4623-966b-88a7925a0ca4
• TSVD-2: ba1ddc3c-65cb-43b8-a150-2bef95a41549
• UMAP-2: da5ef3a6-7158-4d34-8f69-23c12e92bb63
• PCA-16 plus HDBSCAN: 9cdfba96-0299-4220-861c-fef49bf1a52d

-- Project all Win32 challenge histograms to two truncated-SVD components for
-- comparison with cuVS PCA and UMAP.
SELECT p.id,
       p.pc_0,
       p.pc_1,
       v.label,
       m.week_id,
       m.sha256,
       m.family
FROM cuml_tsvd(
    input => (
        SELECT r.id, r.histogram AS vector
        FROM ember_research_vectors_challenge r
        JOIN ember_metadata_challenge m ON m.id = r.id
        WHERE r.split = 'challenge' AND m.file_type = 'Win32'
        ORDER BY r.id
    ),
    n_components => 2
) p
JOIN ember_research_vectors_challenge v ON v.id = p.id AND v.split = 'challenge'
JOIN ember_metadata_challenge m ON m.id = v.id
ORDER BY p.row_ordinal;

Limits

• Validation resolves named tables or views and reads schemas only; it does not execute relation scans or GPU work.
• Execution relation arguments require parenthesized subqueries; dry-run validation accepts registered named relations only.
• Empty input returns an empty result without a native call. Nonempty input requires at least two evaluated rows and two feature dimensions. n_components cannot exceed dimension; known dimensions are checked during planning, otherwise execution reports a structured error.
• Only Float32 features are supported. Use CAST in the input subquery for other element types.
• Component sign is not semantic; this operation returns transformed rows only, not components, explained variance, or mean.
• TSVD allocates a temporary rows * n_components score matrix inside the query device cap.

To dry-run validate relation metadata, column types, and options without execution, see gpu_validate_call.