Integrating Algeon
Algeon runs GPU execution behind ordinary DataFusion APIs: you keep your SQL, DataFrame code, and Arrow results, and choose the integration surface that matches the service boundary you own. Throughout these docs, "native" means Algeon's GPU execution path.
| Surface | Use it when | Start here |
|---|---|---|
| DataFusion session | You already own a Rust backend, DataFusion session lifecycle, auth, tenancy, and result formatting. | DataFusion Session |
| Flight SQL server | You want a remote Arrow Flight SQL endpoint for non-Rust clients, notebooks, agents, or service-to-service calls. | Flight SQL server |
| Embedded backend example | You want one Rust process that serves product-owned REST workflows and an additional Flight SQL endpoint from one shared GPU backend. | Embedded Backend Example |
All three surfaces share one GPU backend. By default, Algeon selects supported
relational SQL for cuDF execution. Choose functions_only to keep ordinary SQL
on DataFusion CPU while explicit GPU functions run on GPU, or native_required
to reject plans containing DataFusion CPU execution. See
Choose an execution mode.
The cugraph
feature makes the cugraph_* table functions executable. The cuvs feature
makes the three documented cuvs_* functions executable once the session
installs cuVS SQL and a backend is configured; see
cuVS execution.
Iceberg support
Iceberg is not supported in this release, and it cannot be enabled from
source: there is no iceberg feature in any manifest. The DataFusion
integration moved out of apache/iceberg-rust into
apache/datafusion-iceberg
(iceberg-rust#3029, merged 2026-09-17). That project already targets
DataFusion 55, but it is not published to crates.io and still pins iceberg by
git rev, so it cannot be referenced from a published manifest. Iceberg support
returns once it ships a release.
Common flow
- Build first. Build the native libraries and then the Rust stack with the features your deployment needs, following Build & Test. Every surface below starts from that build.
- Pick an entry point. Embed a DataFusion session in your Rust service, start the Flight SQL server for remote clients, or run the embedded example that combines both. In each case, declare the GPUs the process may use before running queries; the server-side settings are in Configuration.
- Provide tables. In an embedded session, register tables as you would
in plain DataFusion (
CREATE EXTERNAL TABLE ... STORED AS PARQUETorregister_parquet); see Execute one attempt per query. The server reads local Parquet or Iceberg tables from a REST or Glue catalog configured under Iceberg and workspace. If you use thecugraph_*orcuvs_*functions, list and validate the installed calls with Discover & Validate GPU Functions. - Check whether a query runs on the GPU. Before shipping a query, ask for
its planned execution path, and the reason for any part that stays on the
CPU, with GPU coverage validation:
algeon_explain_coveragein embedded SQL,EXPLAIN GPUthrough Flight SQL, orvalidate_queryin Rust. With the defaultnative_preferredmode, relational candidates not selected for GPU execution stay in DataFusion. - Handle failures. A GPU execution failure is not replayed on CPU; it
comes back as an error with a stable code, kind, and status. At your
application boundary, act on errors that carry a typed Algeon identity and
keep your existing policy for ordinary DataFusion failures. See
Handling Errors. Queries that cannot get GPU
execution capacity in time return
service_overloadedbackpressure rather than waiting indefinitely; waiters are served first-in, first-out.
The cuGraph SQL API and cuVS SQL API sections are the function-level contracts. They share discovery and validation while documenting their distinct relation syntax, input semantics, lifecycle, and execution constraints.