Flight SQL Server
Run the standalone server when clients access Algeon over a remote Arrow Flight
SQL endpoint. The server initializes a shared GPU backend, installs cuGraph,
cuVS, and cuML UDTFs across all sessions (cuGraph can be
disabled in configuration),
and returns Arrow record batches to ADBC and Flight SQL clients. Its
embeddable implementation and public API live in
algeon_datafusion::server; crates/algeon-server is the standalone launcher.
Run the published image
The published image needs no source build. From the directory that holds your Parquet files:
docker run --rm --gpus all -p 127.0.0.1:50051:50051 \
-v "$PWD:/data:ro" ghcr.io/algeon-dev/algeon-server
The image includes cuGraph, cuVS and cuML. Each Parquet file and each
directory of Parquet files under the mounted /data becomes a table at startup
(see Data root). The server listens
on host loopback port 50051 with authentication disabled; continue with the
quickstart. Host requirements, supported GPUs and
troubleshooting are on
Run the published image.
Build and start
To build the server instead, after the native libraries are built (see
Run as a Flight SQL server), build the
algeon-server package and its algeon_server binary:
cargo build --release -p algeon-server --features cugraph,cuvs,cuml
Add nvml to the feature list for GPU utilization, NVML framebuffer and
device identity diagnostics. Query execution and memory admission do not
depend on it; see NVML diagnostics.
Start it with no configuration file:
target/release/algeon_server
With no configuration the server uses GPU 0, binds 127.0.0.1:50051, and
leaves authentication disabled. algeon_server --help lists the command-line
flags, and algeon_server --check-config prints the effective configuration
and the source of each value without touching the GPU. For memory limits,
more GPUs, authentication, and TLS, start from the
complete example.
Before binding another interface, enable
authentication and either
TLS or a TLS-terminating proxy.
To keep ordinary SQL on DataFusion CPU while explicit cuGraph, cuVS, and cuML
calls run on the GPU, set ALGEON_SERVER_NATIVE_EXECUTION_MODE=functions_only
(or execution_mode = "functions_only" under [native]) before startup.
The default is native_preferred. See
Execution mode for all three modes.
From another terminal, verify the endpoint with the repository client:
cargo run -p tools -- \
flight-sql-query grpc://127.0.0.1:50051 "SELECT 1 AS one"
The expected result is a column named one containing 1. The tool is a
plaintext smoke-test client with no authentication or TLS. Applications
connect through ADBC. It waits up to 5 seconds for the
server and prints a progress line each second.
--connect-timeout-secs <n> before the URL changes that limit.
Five-minute quickstart
These statements need no dataset: the GPU functions read inline VALUES
rows. Run them in any client from Client settings.
List the GPU functions this build installed:
SELECT function_name, provider, available, unavailable_reason FROM gpu_list_functions() ORDER BY function_name;
cuml_forest_predict reports available = false with
model_root_unconfigured until the server configuration sets a
[models] root.
Check the server version, enabled features, and execution mode:
SELECT * FROM algeon_version();
Rank four vertices with PageRank. The result has one vertex and value
row per vertex:
SELECT * FROM cugraph_pagerank(edges => (SELECT * FROM (VALUES (1,2),(2,3),(3,1),(3,4)) AS t(src, dst)));
Cluster four points into two groups with cuVS k-means:
SELECT * FROM cuvs_kmeans(input => (SELECT * FROM (VALUES (1, 0.0, 0.0), (2, 0.1, 0.0), (3, 5.0, 5.0), (4, 5.1, 4.9)) AS t(id, x, y)), n_clusters => 2);
gpu_validate_call checks a call against the schema of a registered table
or view. Register the edges as a view, then validate a PageRank call over it:
CREATE OR REPLACE VIEW quickstart_edges AS
SELECT * FROM (VALUES (1,2),(2,3),(3,1),(3,4)) AS t(src, dst);
SELECT *
FROM gpu_validate_call(
'cugraph_pagerank',
'{
"relations":{"edges":{"table":"quickstart_edges"}},
"options":{"alpha":0.9,"directed":true}
}'
);
The row has valid = true and would_execute_gpu = true. Finally, ask the
planner which parts of the PageRank query run on the GPU:
EXPLAIN GPU SELECT * FROM cugraph_pagerank(edges => (SELECT * FROM (VALUES (1,2),(2,3),(3,1),(3,4)) AS t(src, dst)));
The summary row reports gpu_path = native with zero CPU boundaries.
Discover & Validate GPU UDTFs walks
through the same sequence with descriptors and registered data, and
GPU coverage validation explains each
EXPLAIN GPU column.
Start the citation demo server
The citation investigation uses
Parquet files produced by the
citation fixture.
Because cuGraph is enabled by default, the server requires no additional
configuration. Paths in LOCATION resolve against the server's working
directory. Start it from the repository root.
To register the tables at every start, save these statements in a file such
as citation-init.sql:
CREATE OR REPLACE EXTERNAL TABLE citation_edges
STORED AS PARQUET LOCATION 'fixture/demo/citation_network/v1/parquet/edges_by_src.parquet';
CREATE OR REPLACE EXTERNAL TABLE citation_edges_by_dst
STORED AS PARQUET LOCATION 'fixture/demo/citation_network/v1/parquet/edges_by_dst.parquet';
CREATE OR REPLACE EXTERNAL TABLE papers
STORED AS PARQUET LOCATION 'fixture/demo/citation_network/v1/parquet/profiles_by_paper_id.parquet';
CREATE OR REPLACE EXTERNAL TABLE paper_authors
STORED AS PARQUET LOCATION 'fixture/demo/citation_network/v1/parquet/paper_authors.parquet';
CREATE OR REPLACE EXTERNAL TABLE paper_fos
STORED AS PARQUET LOCATION 'fixture/demo/citation_network/v1/parquet/paper_fos.parquet';
and name it with init_sql:
ALGEON_SERVER_INIT_SQL="$PWD/citation-init.sql" target/release/algeon_server
Alternatively, submit each statement separately from a connected SQL client
after startup. The first three tables supply the citation-path, bibliography
and community queries on the investigation page. Examples that use author or
field links also need paper_authors and paper_fos. The saved SQL includes
any additional views that its algorithm inputs require. The default
native_preferred mode supports the bibliography example's mixed execution
path. Inspect a changed query with
GPU coverage validation.
Shared catalog
The DDL namespace is server-wide. A table or view that one connection
creates is visible to every connection and every authenticated user, and a
CREATE OR REPLACE from one client replaces the object for all of them.
When several agents or users share a server, give each one distinct names or
a name prefix. Registrations last until the server stops; init_sql
recreates its objects at the next start.
Statement limits
Flight SQL parameter binding is not supported. A prepared statement with placeholders fails with:
Flight SQL parameter binding is not supported; prepare statements without placeholders
Applications therefore interpolate values into the SQL text. Escape a string value by doubling each single quote, and accept identifiers such as table names only after checking them against an allowlist or a strict pattern:
def sql_string(value: str) -> str:
return "'" + value.replace("'", "''") + "'"
EXPLAIN GPU runs from statement queries and from prepared statements without
parameters; the Python DB-API cursor.execute("EXPLAIN GPU ...") executes
successfully. See GPU coverage validation.
DML and COPY are rejected. A statement query returns
Flight SQL statement query does not support DML or COPY statements, and a
prepared statement returns
Flight SQL prepared DML and COPY statements are not supported.
SET and RESET are rejected on every endpoint with:
SET and RESET are not supported over Flight SQL; configure DataFusion session options with DATAFUSION_* environment variables before starting the server
Set DataFusion session options through
DATAFUSION_* variables
before the server starts.
Connect with ADBC
Use an Apache Arrow ADBC Flight SQL driver from application code. The Python DB-API wrapper requires both the Flight SQL driver and PyArrow:
python -m pip install adbc-driver-flightsql pyarrow
from adbc_driver_flightsql import dbapi
with dbapi.connect(uri="grpc://127.0.0.1:50051") as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1 AS one")
result = cur.fetch_arrow_table()
The DB-API wrapper emits this warning when a connection opens:
Warning: Cannot disable autocommit; conn will not be DB-API 2.0 compliant
Flight SQL does not implement transactions; this warning is expected and requires no action.
The same endpoint works with other ADBC language bindings. Keep pooling, timeouts, authentication, and result conversion in the client service. Query stats describes how to fetch per-query statistics by query id or correlation id.
Connection loss and result backpressure
The server sends an HTTP/2 keepalive ping after 30 seconds without inbound traffic. If an acknowledgement does not arrive within 10 seconds, the server terminates the connection. The same settings apply when the server binds its address and when an embedded caller supplies a prebound listener. Connection loss drops the Flight result stream, cancels its attempt, and releases the GPU grant after native cleanup completes. Each statement registers a cancellation token derived from the server's shutdown token. When it fires, the next poll of the result body cancels the attempt and drops the inner stream (see Completion and resident data).
A client that acknowledges pings but stops reading results keeps its query under output backpressure and retains its grant until the stream ends or is canceled. Result chunks are at most 4 MiB, and unread result chunks trigger HTTP/2 flow control that halts subsequent poll requests. Keepalive sets no query idle deadline. Clients should consume or cancel result streams before returning a connection to their pool.
Use an interactive client
DataBow
databow exercises the ADBC driver
path. Install the Flight SQL driver with
dbc first, then connect:
dbc install flightsql
databow --driver flightsql --uri grpc://127.0.0.1:50051
arrow_cli
arrow_cli connects directly over
Flight SQL:
arrow_cli --host 127.0.0.1 --port 50051 --timeout 120
Client settings
The ADBC option names are those of adbc_driver_flightsql 1.12.0. A client
and server that disagree on TLS fail at connection time:
- A plaintext client against a TLS server reports
error reading server preface: unexpected EOF. Switch the URI togrpc+tls://. - A TLS client against a plaintext server reports a TLS handshake error.
Switch the URI to
grpc://, or enable TLS on the server.