Skip to main content

Configuration

This page is the settings reference for the Flight SQL server process, ordered by TOML table. An embedded Rust service configures the engine through DataFusion Session.

SurfaceConfigure withOwns
Flight SQL processServerConfig, TOML, and environment variablesListen address, admission, auth, TLS, startup SQL, and retention
Embedded GPU backendAlgeonGpuBackend::builder()Device ownership, capacity, queues, and execution mode
DataFusion sessionSessionConfigStandard DataFusion behavior (catalogs, batch size, target partitions)

The server loads its settings in three layers. Built-in defaults come first. The TOML file named by --config or ALGEON_SERVER_CONFIG_FILE replaces any default it sets. Environment variables then override single values. Every setting has one TOML location, and its environment variable is ALGEON_SERVER_ followed by the TOML path in upper case with . written as _: native.execution_mode becomes ALGEON_SERVER_NATIVE_EXECUTION_MODE. Only strings, integers and booleans have an environment form. The [[admission.device_profiles]], [[iceberg.catalogs]] and [[auth.mode.users]] arrays live in the file only. An unknown TOML field stops startup with invalid_configuration and the error lists the table's valid keys; an unknown ALGEON_SERVER_* variable stops startup the same way and names the closest known variable when one is similar.

Two variables have no TOML location because they apply before the file is read: ALGEON_SERVER_CONFIG_FILE and ALGEON_SERVER_LOG (default info).

Minimal server​

No file is needed. A release build starts on GPU 0 with every default and listens on 127.0.0.1:50051:

target/release/algeon_server

Add a file when the deployment needs memory limits, more GPUs, authentication or TLS. algeon_server --check-config prints what the server would run with, without touching a GPU.

Complete example​

Every field with its default and its environment variable. Commented lines are unset by default.

# server.toml
bind = "127.0.0.1:50051" # ALGEON_SERVER_BIND
statement_ticket_ttl_secs = 300 # ALGEON_SERVER_STATEMENT_TICKET_TTL_SECS
http2_keepalive_interval_secs = 30 # ALGEON_SERVER_HTTP2_KEEPALIVE_INTERVAL_SECS
http2_keepalive_timeout_secs = 10 # ALGEON_SERVER_HTTP2_KEEPALIVE_TIMEOUT_SECS
# init_sql = "/etc/algeon/init.sql" # ALGEON_SERVER_INIT_SQL

[admission]
max_queued_attempts = 64 # ALGEON_SERVER_ADMISSION_MAX_QUEUED_ATTEMPTS
max_queue_wait_ms = 30000 # ALGEON_SERVER_ADMISSION_MAX_QUEUE_WAIT_MS

[[admission.device_profiles]] # file only; one entry per GPU
device_ordinal = 0
# memory_limit_bytes = 85899345920 # automatic capacity when unset
backend_reserve_bytes = 0 # required with memory_limit_bytes
# default_query_limit_bytes = 21474836480 # derived from capacity when unset
max_active_attempts = 2
source_work_per_attempt = 1
cache_cap_bytes = 0
install_telemetry = true
# worker_count = 3 # set all three or none
# stream_pool_size = 8
# lane_count = 2

[native]
execution_mode = "native_preferred" # ALGEON_SERVER_NATIVE_EXECUTION_MODE
# max_source_chunk_bytes = 1073741824 # ALGEON_SERVER_NATIVE_MAX_SOURCE_CHUNK_BYTES
max_row_groups_per_chunk = 1 # ALGEON_SERVER_NATIVE_MAX_ROW_GROUPS_PER_CHUNK
max_retry_attempts = 3 # ALGEON_SERVER_NATIVE_MAX_RETRY_ATTEMPTS
max_source_reads_in_flight = 2 # ALGEON_SERVER_NATIVE_MAX_SOURCE_READS_IN_FLIGHT
source_chunk_planning_mode = "parquet_throughput" # ALGEON_SERVER_NATIVE_SOURCE_CHUNK_PLANNING_MODE
statistical_aggregate_execution_mode = "gpu_native_tolerant" # ALGEON_SERVER_NATIVE_STATISTICAL_AGGREGATE_EXECUTION_MODE

[observability]
per_query_max_records = 65536 # ALGEON_SERVER_OBSERVABILITY_PER_QUERY_MAX_RECORDS
per_query_max_bytes = 8388608 # ALGEON_SERVER_OBSERVABILITY_PER_QUERY_MAX_BYTES
# active_logical_capture_max_bytes = 4294967296 # ALGEON_SERVER_OBSERVABILITY_ACTIVE_LOGICAL_CAPTURE_MAX_BYTES
global_sealed_max_count = 1024 # ALGEON_SERVER_OBSERVABILITY_GLOBAL_SEALED_MAX_COUNT
global_sealed_max_bytes = 1073741824 # ALGEON_SERVER_OBSERVABILITY_GLOBAL_SEALED_MAX_BYTES
sealed_ttl_secs = 3600 # ALGEON_SERVER_OBSERVABILITY_SEALED_TTL_SECS
shutdown_grace_secs = 30 # ALGEON_SERVER_OBSERVABILITY_SHUTDOWN_GRACE_SECS

[cugraph]
enabled = true # ALGEON_SERVER_CUGRAPH_ENABLED
construction_policy = "python_cugraph" # ALGEON_SERVER_CUGRAPH_CONSTRUCTION_POLICY
max_input_staging_bytes = 1073741824 # ALGEON_SERVER_CUGRAPH_MAX_INPUT_STAGING_BYTES
max_output_staging_bytes = 1073741824 # ALGEON_SERVER_CUGRAPH_MAX_OUTPUT_STAGING_BYTES

[models]
# root = "/srv/models" # ALGEON_SERVER_MODELS_ROOT
# max_model_bytes = 268435456 # ALGEON_SERVER_MODELS_MAX_MODEL_BYTES

[data]
# root = "/data" # ALGEON_SERVER_DATA_ROOT

# [s3] # absent: s3:// locations fail
# endpoint = "http://127.0.0.1:9000" # ALGEON_SERVER_S3_ENDPOINT
# region = "us-east-1" # ALGEON_SERVER_S3_REGION
# path_style = true # ALGEON_SERVER_S3_PATH_STYLE
# unsigned = false # ALGEON_SERVER_S3_UNSIGNED

# [[iceberg.catalogs]] # file only; requires [s3]
# name = "lake"
# kind = "rest"
# uri = "http://127.0.0.1:8181"
# auth = { kind = "none" }

[auth.mode]
kind = "disabled" # ALGEON_SERVER_AUTH_MODE_KIND
# token_ttl_secs = 3600 # ALGEON_SERVER_AUTH_MODE_TOKEN_TTL_SECS

# [[auth.mode.users]] # file only
# username = "alice"
# password_hash = "$argon2id$v=19$m=19456,t=2,p=1$..."

# [tls]
# cert_path = "/etc/algeon/server.pem" # ALGEON_SERVER_TLS_CERT_PATH
# key_path = "/etc/algeon/server.key" # ALGEON_SERVER_TLS_KEY_PATH
# client_ca_path = "/etc/algeon/clients-ca.pem" # ALGEON_SERVER_TLS_CLIENT_CA_PATH

The commented byte values in [[admission.device_profiles]] and [observability] illustrate the units. Their defaults depend on the GPU.

Server​

FieldDefaultPurpose
bind127.0.0.1:50051Flight SQL listen address. The default accepts local clients only
statement_ticket_ttl_secs300How long an unconsumed statement ticket waits for DoGet
http2_keepalive_interval_secs30Seconds without inbound HTTP/2 traffic before a keepalive ping
http2_keepalive_timeout_secs10Seconds to acknowledge a ping before the connection closes
init_sqlunsetSQL file of DDL statements run at startup (Startup SQL)

bind takes an IP socket address such as 127.0.0.1:50051 or 0.0.0.0:50051. Host names are not resolved. The durations must be positive. A client that acknowledges keepalive pings can retain an unread result. These durations do not impose a query idle deadline. Embedders set ServerConfig::http2_keepalive through Http2Keepalive.

The server binds its port after the GPU backend and the session are ready, so an open port means the server accepts queries. The starting Algeon Flight SQL server log line follows a successful bind. When the port is taken, startup fails with server_transport_failure and failed to bind Flight SQL listener at <address>.

Admission​

FieldDefaultPurpose
admission.max_queued_attempts64Maximum queued admission waiters
admission.max_queue_wait_ms30000Maximum wait for a compatible device
  • Admission is FIFO among capability-compatible waiters, with no overtaking or protected-waiter setting.
  • The queue wait ends when a device grant is placed. The query's own deadline and cancellation bound the execution-context build after that.
  • Capabilities and memory ownership are not file settings. The shipped server declares WholeDeviceExclusive ownership for every selected GPU; custom embedders declare it through the Rust API.

The selected GPUs are the device_ordinal values of the [[admission.device_profiles]] entries, in file order. Without any entry the server selects GPU 0 with a profile that sets nothing, and the startup log says the profile was defaulted. Each entry takes the fields below. A rejected value stops startup with invalid_configuration.

FieldDefaultConstraint
device_ordinalRequiredOne entry per GPU. A second entry for the same ordinal is an error.
memory_limit_bytesAutomatic capacityAt most the device memory Algeon owns.
backend_reserve_bytes0Required with an explicit memory_limit_bytes and must leave managed capacity.
default_query_limit_bytesDerived from managed capacityPositive and within managed capacity minus cache_cap_bytes.
max_active_attempts2Positive.
source_work_per_attempt1Positive.
cache_cap_bytes0 (no resident GPU cache)Below managed capacity.
install_telemetrytrueOptional boolean.
worker_countmax_active_attempts + 1At least max_active_attempts + 1.
stream_pool_size, lane_countEngine-derivedSet together with worker_count, or omit all three.
  • worker_count counts the device's GPU execution workers. One is reserved for context setup and release cleanup, letting queries start and finish while the others execute queries.
  • Automatic capacity already subtracts a driver reserve. An explicit limit declares its own backend_reserve_bytes, where 0 is a deliberate choice.
  • The shipped server owns the whole device. Its memory_limit_bytes ceiling is the CUDA-visible total, below the nominal card size.
  • default_query_limit_bytes is the immutable cap for a query whose embedding caller supplies none. Planner estimates do not resize it.
  • After the backend starts, the log prints one line per GPU with its managed capacity, effective default_query_limit_bytes, max_active_attempts and worker_count.
  • A rejected capacity prints the setting, configured value, ceiling and largest accepted value in bytes and GiB. Each of the four byte fields reports the largest value its capacity accepts. Lower the setting in that device's entry to at most that value.
Example rejection
invalid server configuration: failed to initialize ownership-enforcing GPU backend core: device 0 memory_limit_bytes is 102005473280 bytes (95.00 GiB) but Algeon owns 101973819392 bytes (94.97 GiB) of device memory; the largest accepted memory_limit_bytes is 101973819392 bytes (94.97 GiB) [device_ordinal=0, configured_limit_bytes=102005473280, owned_capacity_bytes=101973819392, reason=explicit_capacity_exceeds_owned_capacity]

Native execution​

Execution mode​

The default is native_preferred. It selects supported relational SQL for native GPU execution and retains DataFusion for unsupported or unselected candidates. Set it in the configuration file or environment:

[native]
execution_mode = "native_preferred"
export ALGEON_SERVER_NATIVE_EXECUTION_MODE=native_preferred

Choose an execution mode describes native_preferred, functions_only and native_required, their composition boundaries, host staging limits and embedded Rust configuration. The mode is backend policy with no SQL SET form. Every mode still needs UDTF features, registration, device profiles and GPU admission. SELECT * FROM algeon_version() returns the running mode with the version, commit and enabled features.

Native settings​

[native] fieldDefault
execution_modenative_preferred
max_source_chunk_bytesUnset: derived from the admitted grant
max_row_groups_per_chunk1
max_retry_attempts3
max_source_reads_in_flight2
source_chunk_planning_modeparquet_throughput (or conservative)
statistical_aggregate_execution_modegpu_native_tolerant (or datafusion_exact_host_welford)
  • Device capacity comes only from [[admission.device_profiles]]. Native settings carry no device-budget planning placeholder.
  • Without max_source_chunk_bytes, the immutable grant divided by the chunk regions the consuming task keeps live sets the source-byte target. An explicit value up to the grant lowers that target, and a larger one replaces it. The startup log prints auto for an unset value.
  • Both targets bound a chunk's decoded size, estimated from the Parquet footer. A dictionary-encoded string column counts at its decoded width, not its encoded size.
  • parquet_throughput lets one chunk of a Parquet file hold up to 1,024 row groups within the target when max_row_groups_per_chunk is 1. It applies to local files and to s3:// objects whose footers planning read, and needs no remote-specific setting: on a g7e.8xlarge reading same-region S3, a SUM over the 17.2 GB TPC-DS SF100 store_sales file takes 1.1 s against 270 s with one row group per chunk. A chunk never spans files, so a table of many small remote files reads one chunk per file. Other sources keep one row group per chunk. conservative plans max_row_groups_per_chunk row groups per chunk; compatible chunks then coalesce into batched reads under the same target.
  • When the grant rejects an allocation while a chunk is read, staged through its source-adjacent filters and projections or partitioned for an aggregate, the chunk splits in half by row group and runs again, up to max_retry_attempts attempts.
  • Before the next read, source-adjacent aggregates account for observed decoded and staged bytes per row and live allocation-domain usage. The allocator enforces the grant, retained aggregate and graph state included.

statistical_aggregate_execution_mode governs stddev and var. The default keeps them on the GPU, where floating-point bits can differ from DataFusion and precision degrades when values are large relative to their spread. datafusion_exact_host_welford matches DataFusion by copying the aggregate input to host memory. A query is not rewritten to native execution when that mode has no exact evaluation for its group keys or the other aggregates in the same GROUP BY.

Source reads use KvikIO with compatibility mode ON, 64 I/O threads and 4 MiB tasks, set once when the backend is built. KvikIO's own variables KVIKIO_COMPAT_MODE, KVIKIO_NTHREADS and KVIKIO_TASK_SIZE override them, and no [native] field sets them. The startup log line source I/O ready prints the effective values. The data path describes the shared thread pool and its pinned host memory.

Authentication​

Authentication defaults to disabled. Use that default only on a trusted local network. Static Basic credentials can be exchanged for server-issued bearer tokens:

[auth.mode]
kind = "basic_bearer"
token_ttl_secs = 3600

[[auth.mode.users]]
username = "alice"
password_hash = "$argon2id$v=19$m=19456,t=2,p=1$..."

token_ttl_secs is required and positive. Non-expiring bearer tokens are unsupported. password_hash must be a valid Argon2 PHC string, plaintext is rejected, and usernames must be unique. Generate a hash with the server binary. It reads one line from stdin and prints the PHC string:

printf '%s\n' 'correct horse battery staple' | target/release/algeon_server hash-password

Run interactively, hash-password prompts on stderr. Because the standard library cannot mask terminal input, typed characters remain visible on screen. Piping standard input prevents displaying the password.

TLS​

The listener serves plaintext by default. A [tls] table switches it to TLS with PEM files:

[tls]
cert_path = "/etc/algeon/server.pem" # certificate chain, leaf first
key_path = "/etc/algeon/server.key" # matching private key
client_ca_path = "/etc/algeon/clients-ca.pem" # optional
  • cert_path and key_path are required together.
  • With client_ca_path, the server requires every client to present a certificate that chains to that CA (mutual TLS). Without it, clients are not asked for one.
  • Startup fails with invalid_configuration before GPU initialization when a file cannot be read, a PEM is invalid, or the key does not match the certificate. An unreadable file is named by its field and path. Key material never appears in the error.
  • Files are read once during process startup; certificate renewals require restarting the server.
  • Clients connect with grpc+tls://host:50051 and trust the CA that signed the server certificate.
  • Loading [tls] logs client_certificate_verification, and the starting Algeon Flight SQL server line reports tls_enabled.
  • Embedded Rust sets ServerConfig::tls to a tonic ServerTlsConfig that carries an identity.

Basic credentials and bearer tokens travel in cleartext unless the connection uses TLS or a TLS-terminating proxy. A deployment behind such a proxy or a service-mesh sidecar leaves [tls] unset and binds the interface the proxy reaches. The server does not refuse a plaintext bind on a non-loopback address.

Runtime observation retention​

The defaults retain bounded query records and sealed reports. Change them only when traffic or exporter behavior needs another budget. A data statement's runtime observation belongs to its execution attempt. A metadata statement, such as a catalog query, runs no native work. Its runtime artifact is the control frame, and its query stats come from the server's own capture.

[observability] fieldDefault
per_query_max_records65536
per_query_max_bytes8 MiB
active_logical_capture_max_bytesUnset: admission requirement plus 1 GiB
global_sealed_max_count1024
global_sealed_max_bytes1 GiB
sealed_ttl_secs3600
shutdown_grace_secs30

All values must be positive. The capture budget never refuses a query. One that does not fit runs without capture, its SQL text is not kept, and its query stats report unavailable_reason=active_capture_budget during and after the run. Admission alone decides overload.

How the capture budget is computed

Each live Flight SQL query reserves logical capture bytes until it finishes: three times per_query_max_bytes plus its sealed artifact (twice per_query_max_bytes, capped at global_sealed_max_bytes), 40 MiB with the defaults. It holds the reservation while planning and finalizing, outside its admission slot.

Unset, the budget is the capture of every attempt admission can hold at once (each profile's max_active_attempts plus admission.max_queued_attempts) plus 1 GiB. The server logs the budget and that requirement at startup.

Reservations follow registration order. Statements that plan, finalize or never take an admission slot also hold them, and a query within the admission bounds can still run without capture.

The server registers at most 1024 live Flight SQL statements, counting every statement that is planning, ticketed but unread, executing (CPU-only and metadata included) or finalizing. Startup and FlightSqlServerBuilder::build fail with admission_exceeds_live_query_capacity when the admission bounds alone hold 1024 attempts or more. Its config_field is admission.max_queued_attempts, or admission.device_profiles.max_active_attempts when the profiles fill the cap alone. Because statements outside admission share the cap, registration can refuse one while admission has room.

GPU SQL UDTFs​

[cugraph] fieldDefaultMeaning
enabledtrueRegisters the cugraph_* SQL frontend.
construction_policypython_cugraphpython_cugraph or raw_libcugraph.
max_input_staging_bytes1 GiBRetained Arrow batches from edges and graph side inputs, counted together.
max_output_staging_bytes1 GiBEagerly materialized Arrow results.
  • With enabled = false, any other [cugraph] setting fails startup. Staging values must be positive.
  • A binary without the cugraph feature can expose the frontend, but execution returns a structured required_feature_disabled error.
  • Per-call graph values, such as a named construction_policy, override these defaults (Graph Inputs and Construction).
  • Input buffers shared by slices or by different source partitions count once across retained batches. GPU-resident inputs bypass host input staging.
  • Staging limits apply to host RAM apart from the device grant, and the DataFusion memory pool may impose a lower limit.
  • Embedded sessions pass GpuIslandHostBoundaryConfig through CugraphSqlConfig.host_boundary_config. Conflicting reinstallations fail.

cuVS SQL has no enable setting and is installed on every session. The cuvs feature decides whether its UDTFs execute or return required_feature_disabled (cuVS execution).

Models and forest inference​

[models]
root = "/srv/models"
max_model_bytes = 268435456
[models] fieldDefaultConstraint
rootunsetEnables cuml_forest_predict and must be an existing directory at startup.
max_model_bytes256 MiBPositive, and only valid with root.

Without root, gpu_list_functions() lists cuml_forest_predict as unavailable with model_root_unconfigured. Embedded sessions use CumlModelStoreConfig through try_with_cuml_sql_config, and Model files covers path resolution, rejections and the trust boundary.

Data root​

[data]
root = "/data"

root names a directory whose Parquet data the server registers as tables before it runs startup SQL. It must be an existing directory when the configuration loads. Unset, no tables are registered. Only the first level of the directory is read, in name order:

EntryBecomes
orders.parquet (extension in any case)Table orders over that file
Directory events/ that holds Parquet filesTable events, as CREATE EXTERNAL TABLE events STORED AS PARQUET LOCATION 'file:///data/events/' would register it
A name starting with . or _Skipped, so Spark's _SUCCESS markers and hidden files are ignored
Any other file, or a directory without Parquet filesSkipped

Table names keep the case and every character of the file or directory name, including glob characters such as [, * and ?. Query Orders.parquet as "Orders" and sales[2024].parquet as "sales[2024]".

The server logs a warning naming the path and skips:

  • an entry it cannot inspect, such as a symbolic link whose target is outside a container bind mount;
  • an entry whose name is not valid UTF-8;
  • a directory whose listing fails, for example on a subdirectory without read permission, before a Parquet file is found.

These stop startup with invalid_configuration naming the paths:

  • two entries that map to one table name, such as sales.parquet and sales/;
  • a Parquet file or a directory holding Parquet files whose schema cannot be read or that cannot be registered, with the reader error in the message.

The log prints one registered data root table line with the table name and path for each table, then the total count. The tables live in the shared catalog. Startup SQL can build views over them or replace one with CREATE OR REPLACE EXTERNAL TABLE.

S3 storage​

[s3]
endpoint = "http://127.0.0.1:9000"
region = "us-east-1"
path_style = true

An [s3] table lets CREATE EXTERNAL TABLE, startup SQL and queries read s3:// Parquet locations. Without the table those locations fail. An empty [s3] enables them with the defaults below, and so does setting any ALGEON_SERVER_S3_* variable. Native reads, footer pruning and DataFusion scans of s3:// sources all use these settings.

[s3] fieldDefaultEnvironment variable
endpointUnset: AWS S3, or the endpoint the AWS environment namesALGEON_SERVER_S3_ENDPOINT
regionUnset: AWS_REGION, then AWS_DEFAULT_REGION, then us-east-1ALGEON_SERVER_S3_REGION
path_styleUnsetALGEON_SERVER_S3_PATH_STYLE
unsignedfalseALGEON_SERVER_S3_UNSIGNED
  • Credentials come from the default AWS credential chain: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN, a web identity token, then container or EC2 instance profile credentials. The table has no key fields.
  • unsigned = true sends anonymous requests and reads no credentials, for public buckets.
  • path_style = true addresses objects as <endpoint>/<bucket>/<key>; false selects virtual-hosted addressing (<bucket>.<endpoint>/<key>).
  • A custom endpoint, such as MinIO or RustFS, is read with path-style addressing only. Startup and --check-config fail with invalid_configuration on s3.path_style when endpoint is set together with path_style = false. Set path_style = true or leave it unset.

Embedded sessions pass the same settings as an S3SourceStorageProfile to AlgeonNativeExecutionConfig::with_s3_storage.

Iceberg catalogs​

[s3]
region = "eu-west-2"

[[iceberg.catalogs]]
name = "lake"
kind = "rest"
uri = "http://127.0.0.1:8181"
warehouse = "s3://warehouse/"
auth = { kind = "oauth2", credential = "client-id:client-secret" }

[[iceberg.catalogs]]
name = "glue"
kind = "glue"
warehouse = "s3://bucket/warehouse"

Each [[iceberg.catalogs]] entry names an Iceberg catalog that becomes a DataFusion catalog called name. A server binary reads these catalogs only when it is built with the iceberg feature; a server without it refuses a non-empty [[iceberg.catalogs]] at startup. Supported catalogs, scan behavior and rejected table features are in Iceberg support.

KeyKindDefaultMeaning
nameBothRequiredDataFusion catalog name
kindBothRequiredrest or glue
urirestRequiredCatalog base URI
warehouserestUnsetWarehouse identifier sent to the catalog
prefixrestUnsetPath prefix after the catalog's /v1 segment
authrest{ kind = "none" }Catalog authentication, below
warehouseglueRequiredWarehouse location, such as s3://bucket/warehouse
catalog_idglueUnset: the caller's AWS accountGlue catalog account ID
regionglueUnset: the AWS environmentGlue service region
endpointglueUnset: the AWS Glue endpointGlue service endpoint override
auth.kindKeys
noneNone. Requests carry no authorization
oauth2credential (required), as client_id:client_secret or a client secret alone; server_uri, the token endpoint, defaulting to the catalog's own; scope
tokentoken (required), a bearer token sent with every request

A Glue catalog uses the default AWS credential chain described under S3 storage.

The iceberg table is file-only. An array of tables has no environment form, and no ALGEON_SERVER_ICEBERG_* variable exists. credential and token can therefore be set only in the file. --check-config prints # credential is set or # token is set in their place, and the Debug output of ServerConfig prints <redacted>.

Startup and --check-config stop with invalid_configuration when:

  • an entry is present without an [s3] table, which the server reads Iceberg data files through (reason iceberg_catalogs_require_s3);
  • an entry lacks a required key, or holds a key that belongs to the other kind, to another auth.kind, or to neither, such as a property map;
  • two entries share a name (duplicate_iceberg_catalog), or a name is empty or datafusion, the default DataFusion catalog (iceberg_catalog_name_reserved);
  • a Glue entry sets endpoint while [s3] names no endpoint (iceberg_glue_endpoint_requires_s3_endpoint). iceberg-rust's Glue catalog then reads table metadata through the Glue endpoint, so set s3.endpoint, for example to the regional S3 endpoint.

Embedded Rust reads the validated entries from ServerConfig::iceberg_catalogs.

Startup SQL​

init_sql names a SQL file that the server executes on its shared session before binding the network listener. This pre-populates tables and views without requiring client DDL after restarts:

-- /etc/algeon/init.sql
CREATE EXTERNAL TABLE lineitem STORED AS PARQUET LOCATION '/data/tpch/sf1/lineitem.parquet';
CREATE VIEW recent_lineitem AS SELECT * FROM lineitem WHERE l_shipdate >= DATE '1998-01-01';
  • Statements run one at a time in file order, and the log names each object created.
  • Each statement must be DDL that a Flight SQL client may also send: CREATE [OR REPLACE] EXTERNAL TABLE, CREATE VIEW, CREATE SCHEMA and their DROP forms. CREATE TABLE, CREATE TABLE AS SELECT, a query, INSERT, SET or any other statement stops startup.
  • A parse error, a rejected statement or a failing statement stops startup with invalid_configuration. The message names init_sql, the 1-based statement number and the first 120 characters of the statement, with OPTIONS values masked.
  • Configuration validation verifies that the file exists; --check-config reports an error when the path cannot be resolved. Relative paths inside the SQL resolve against the server's working directory.
  • Objects created here share the server-wide namespace with client DDL.

Command line​

algeon_server [--config <path>] [--log <filter>] [--check-config]
algeon_server --version
algeon_server --help
algeon_server hash-password
ArgumentEffect
--config <path>TOML file. Overrides ALGEON_SERVER_CONFIG_FILE
--log <filter>Tracing filter such as info or algeon_datafusion=debug. Overrides ALGEON_SERVER_LOG. Server log lists the fields of a failure event
--check-configLoads and validates the configuration, including the TLS files, without touching a GPU, prints it, exits 0, or prints the error and exits 1
--versionPrints the version, commit, enabled features and DataFusion version
hash-passwordReads one line from stdin and prints its Argon2 PHC string (Authentication)

An unknown argument prints the usage to stderr and exits with status 2.

--check-config prints the effective configuration as TOML on stdout. Each value carries its source, and log lines go to stderr:

$ ALGEON_SERVER_NATIVE_EXECUTION_MODE=native_required target/release/algeon_server --check-config
# Effective Algeon server configuration. Config file: none
bind = "127.0.0.1:50051" # source: default
http2_keepalive_interval_secs = 30 # source: default
http2_keepalive_timeout_secs = 10 # source: default
# init_sql is unset # source: default
statement_ticket_ttl_secs = 300 # source: default

[admission]
max_queue_wait_ms = 30000 # source: default
max_queued_attempts = 64 # source: default

[[admission.device_profiles]] # defaulted
device_ordinal = 0 # source: default
...

[native]
execution_mode = "native_required" # source: env ALGEON_SERVER_NATIVE_EXECUTION_MODE
max_retry_attempts = 3 # source: default
max_row_groups_per_chunk = 1 # source: default
# max_source_chunk_bytes is unset # source: default
...

At startup the log prints the version, commit, features, config file, listen address, TLS and auth state, execution mode and the other effective settings, then one server setting differs from its default line for each value that is not a default, with its source.

DataFusion session configuration​

The server reads SessionConfig::from_env(), with keys such as DATAFUSION_EXECUTION_BATCH_SIZE, DATAFUSION_EXECUTION_TARGET_PARTITIONS, DATAFUSION_CATALOG_DEFAULT_CATALOG and DATAFUSION_CATALOG_DEFAULT_SCHEMA. In native_preferred and native_required, the native optimizer overrides one DataFusion optimizer default, described in Uncorrelated scalar subqueries.

SET and RESET are rejected over Flight SQL. Session options come from DATAFUSION_* variables before startup.