Introducing AlgeonInteractive Graph, Vector, and ML Analytics for the Agent's Loop
Which brands moved toward Barbie's audience around the July 2023 film release? An agent put that question to Algeon over Arrow Flight SQL, using MerRec, a dataset of 1.27 billion marketplace events. The query compared brand affinity in May-June with July-August. Primark and AMC appeared near the top, alongside the cosmetics brand NYX in first place and Taylor Swift in fifth. What could explain those changes?
Algeon runs graph, vector, and machine-learning algorithms as table functions inside SQL. Each function takes rows from a subquery and returns rows for the rest of the query to use. In this investigation, SQL selected the likes, graph algorithms measured brand affinity, and a final join attached brand names to the ranking.
The ranking also raised questions that needed different SQL. An endpoint built to return brand rankings might accept a new date range or seed brand, but checking the titles of liked products requires another query. With Algeon, an agent or analyst can express that follow-up through the SQL interface, using the tables and functions available to the session.
Replacing glue code with in-plan UDTFs
A workflow built from separate SQL and graph tools often exports the selected rows, constructs a graph in application code, runs an algorithm, and imports the scores for another join. Algeon's user-defined table functions (UDTFs) put these dependencies in the query plan. The query author supplies the input relations and algorithm parameters; the engine executes the operators and manages their memory.
For the Barbie query, each period has a graph connecting shoppers to the brands they liked. Personalized PageRank starts from the Barbie vertex and scores connections through that graph. Dividing each score by ordinary PageRank adjusts for the brand's overall graph score. We call this ratio its affinity to Barbie's audience.
The query compares affinity across the periods, keeps brands with at least 1,000 distinct fans in both, and ranks the relative change. The July-August window starts on July 1, twenty days before the film opened, so the comparison includes the weeks leading up to the release. The SQL preview ends after the four graph calls.
WITH likes AS (
SELECT DISTINCT -(e.user_id + 1) AS src, CAST(i.brand_id AS BIGINT) AS dst,
e.event_time >= TIMESTAMP '2023-07-01 00:00:00' AS after_release
FROM merrec_events e JOIN merrec_items i ON i.item_id = e.item_id
WHERE e.event_type = 'item_like' AND i.brand_id <> 0
AND e.event_time >= TIMESTAMP '2023-05-01 00:00:00'
AND e.event_time < TIMESTAMP '2023-09-01 00:00:00'
), lift_before AS (
SELECT p.vertex, p.value / r.value AS lift
FROM cugraph_personalized_pagerank(
edges => (SELECT src, dst FROM likes WHERE NOT after_release),
personalization => (SELECT 2595 AS vertex, 1.0 AS value),
directed => false) p
JOIN cugraph_pagerank(
edges => (SELECT src, dst FROM likes WHERE NOT after_release), directed => false) r
ON r.vertex = p.vertex
), lift_after AS (
SELECT p.vertex, p.value / r.value AS lift
FROM cugraph_personalized_pagerank(
edges => (SELECT src, dst FROM likes WHERE after_release),
personalization => (SELECT 2595 AS vertex, 1.0 AS value),
directed => false) p
JOIN cugraph_pagerank(
edges => (SELECT src, dst FROM likes WHERE after_release), directed => false) r
ON r.vertex = p.vertex
), fans AS (
SELECT dst AS brand_id,
SUM(CASE WHEN after_release THEN 0 ELSE 1 END) AS fans_before,
SUM(CASE WHEN after_release THEN 1 ELSE 0 END) AS fans_after
FROM likes GROUP BY dst
), brands AS (
SELECT CAST(brand_id AS BIGINT) AS brand_id, MIN(brand_name) AS brand_name
FROM merrec_items WHERE brand_id <> 0 GROUP BY brand_id
)
SELECT b.brand_name, f.fans_before, f.fans_after,
lb.lift AS lift_before, la.lift AS lift_after, la.lift / lb.lift AS lift_change
FROM lift_after la
JOIN lift_before lb ON lb.vertex = la.vertex
JOIN fans f ON f.brand_id = la.vertex
JOIN brands b ON b.brand_id = la.vertex
WHERE la.vertex > 0 AND la.vertex <> 2595
AND f.fans_before >= 1000 AND f.fans_after >= 1000
ORDER BY lift_change DESC, b.brand_name
LIMIT 25;
The ranking put NYX first, followed by AMC, Primark, ALDO, and Taylor Swift.
The graph inputs are subqueries in the plan. Changing a date filter or seed requires editing the SQL. The like graphs and intermediate results remain in GPU memory during this query; graph construction is included in the measured time.
The ranking returned 25 brands in 3.32 seconds in the session replay. In separate repeated runs, its median was 3.05 seconds after the first run. An equivalent CPU workflow using 32-thread DuckDB and igraph took a median of 23.3 seconds on the same host and returned identical top-25 rankings. The case study includes the timing breakdown.
GPU timings in this post use an NVIDIA RTX PRO 6000 with a 90 GiB query
grant and native_required mode. Source Parquet files were staged on local
NVMe and held in the OS page cache for both the GPU and CPU comparisons.
Testing follow-up hypotheses on the working set
A change in graph affinity leaves its cause open. Barbie merchandise could account for some of the ranking. Broader summer trends or an overlapping collector audience could also contribute. We followed the ranking with a product-title check and two reruns using other seed brands. These were separate queries after the recorded agent session.
The title check counts likes on listings whose names contain "Barbie" over the two periods. For NYX, that share rose from 1.83% to 30.62%, with listings such as "LIMITED EDITION NYX Barbie Flip Phone Mirror". For Taylor Swift, only 135 of 164,499 July-August likes went to matching listings.
These results support a merchandise explanation for NYX, Primark, ALDO, and AMC. The name match is an imperfect measure: seller descriptions and color names can also contain "Barbie". It does not establish what caused the graph affinity to change.
Next, we reran the graph query with Nike and Hot Wheels as seeds, changing the seed brand id and its exclusion from the results. None of those four brands appeared among Nike's 25 largest affinity changes. With Hot Wheels, NYX, ALDO, and Primark appeared with smaller changes; AMC was absent from the top 25. The Hot Wheels result is consistent with an overlapping Mattel collector audience. These comparisons help distinguish the rankings, though absence from a top-25 list cannot rule out a broader seasonal change.
The Hot Wheels and Nike reruns took 3.03 and 3.02 seconds respectively, using medians after the first run. The full investigation contains the follow-up SQL and all result rows. The checks support some explanations and leave Taylor Swift's rise unresolved. Investigating that connection would require another question and another query.
Chaining multiple algorithms in one query
The Barbie investigation changed the query as new questions arose. A single question can also require several algorithms. To select 5,000 shoppers for a Pokemon promotion, the campaign query combines graph affinity, vector similarity, and model scores.
Personalized PageRank from the Pokemon brand identifies graph candidates. Exact nearest-neighbor search finds eligible shoppers with tastes similar to historical buyers. An offline-trained XGBoost model ranks the union of these candidates, and the query keeps the top 5,000. A final aggregation counts how many bought Pokemon products in the following two months.
For comparison, the model also ranks all eligible shoppers and those who previously viewed a Pokemon listing. Each pool contributes its top 5,000.
The graph-or-taste shortlist contained 21 more later buyers than the previous-viewer shortlist. Its candidates need not have viewed a Pokemon listing before September. This is a historical comparison; an online randomized trial would be needed to measure whether sending the promotion caused additional purchases.
The query ran in a median of 8.15 seconds after the first run. Intermediate results stayed in GPU memory across cuGraph, cuVS, and cuML, including the candidate relations used by the model and the scores consumed by the final SQL.
A working set for each question
Both examples select algorithm inputs from a larger dataset. The Barbie query builds graphs from distinct shopper-brand likes in each period. The campaign query restricts vector search to eligible shoppers. SQL expresses those filters and joins before the algorithm runs, forming the working set for that question.
Memory requirements include graph construction, algorithm scratch space, and intermediate results as well as the input rows. Across the recorded runs, peak query allocations were 29.70 GiB for Barbie and 44.92 GiB for the campaign, within the configured 90 GiB grant.
Algeon currently runs each query on one GPU and does not spill to disk. A query whose allocations exceed its grant fails. Filters and candidate limits therefore affect both the analytical question and whether it fits the available memory.
Built for the agent's loop
The opening session shows how an agent can discover and use this SQL
interface. It made five Flight SQL calls: two catalog reads for graph
functions and their signatures, a brand-id lookup, an EXPLAIN GPU check
for native coverage, and the ranking query.
The agent wrote the statements in order after reading each response on October 12. The timings above come from replaying those statements on October 15 against an updated server. The session record documents the replay method and results.
Errors also provide information for revising a query. A separate probe asked
for a score column when cugraph_personalized_pagerank returns vertex
and value. Flight returned this error in
3 milliseconds:
Flight returned invalid argument error, with message: No field named score. Valid fields are vertex, value.
The schema check returned valid column names during planning, before GPU execution or device buffer allocation. The calling application can use those names to revise the query.
The application remains responsible for choosing questions, generating SQL, and evaluating the results. Its domain mapping, or ontology, connects terms such as "brand" and "shopper" to tables, keys, and predicates. When several engines serve the lake, the application also tracks their available functions and chooses where to submit each query.
Algeon combines Apache DataFusion with NVIDIA RAPIDS. DataFusion parses SQL
and optimizes the plan. Algeon executes supported relational operators and
cuGraph, cuVS, and cuML table functions on
the GPU. Native stages can keep intermediate relations there until the
final rows return as Arrow record batches. Execution mode controls whether
unsupported operators stay on DataFusion's CPU path or cause planning to
fail; the examples here require native execution. Each GPU query attempt
receives a fixed memory grant, enforced during execution.
Beside the engines already on the lake
Algeon targets investigations over prepared data. In an existing deployment, Spark jobs can continue preparing Parquet tables, while engines such as Trino, ClickHouse, or StarRocks serve the application's standard analytical queries. The application can submit graph, vector, and ML queries to Algeon against those Parquet files.
This arrangement requires data preparation. The examples use local Parquet copies on the GPU host. Algeon does not provide Trino or ClickHouse connectors, and it does not resolve Iceberg snapshots from table metadata. Adapters for query engines other than DataFusion are not planned. The current storage and execution limits are listed below.
Known gaps
Next
The Barbie investigation illustrates the workflow Algeon supports today. A graph ranking led to a product-title check and comparisons with other seed brands, all expressed through SQL. Some results gained supporting evidence; Taylor Swift remained a question for further investigation. The campaign example extended that interface to graph, vector, and model operations within one query. A later post puts a decision model's answers back into the bridges and brand twins queries and checks the rebuilt rankings against controls.
Algeon is available to try when the required data can be staged as Parquet and the query fits on one GPU. Running it currently requires Linux, an NVIDIA GPU, and a source build. Start with the MerRec setup to reproduce these queries and measurements. The development plan includes closer sizing of memory grants and packaged releases that remove the native build requirement. Issues and operator coverage requests are tracked on GitHub.