feat(appkit): support Genie agent mode with reasoning and visualizations by fjakobs · Pull Request #732 · databricks/appkit

fjakobs · 2026-11-12T17:51:38Z

Problem

The genie plugin only uses the Genie chat (conversation) API. That API doesn't return the agent's reasoning, and AppKit dropped the charts Genie draws (viz attachments). Genie agent mode streams the agent's reasoning and renders its charts, but through a separate API that AppKit didn't call.

What the agent API returns

I checked these calls against a real Genie agent:

POST /api/2.0/genie/agents/{id}/responses (with enable_viz: true) returns an SSE stream of Responses-style items:
- response.created, which carries the message and conversation ids
- reasoning items
- tool calls (execute_sql, generate_visualization)
- the assistant message
- response.completed
Items arrive whole; there are no text deltas. A follow-up sends conversation_id in the request body.
The completed message (GET /api/2.0/genie/spaces/{id}/conversations/{cid}/messages) has the query and viz attachments, but no answer text and no reasoning. Those are only in the stream and in GET /api/2.0/genie/agents/{id}/conversations/{cid}/items.
GET …/attachments/{viz_attachment_id}/download-visualization returns the PNG. Passing the query attachment id instead returns a 404.
Any Genie space can be used as an agent, since the agent id is the space id. The SDK covers download-visualization, but not the responses or items endpoints, so those go through workspaceClient.request() and the existing readSseEvents.

Fix

Agent mode (first commit)

Config: a new agents map, with the same shape as spaces:
genie({ spaces: { sales: id }, agents: { analyst: id } }). Agent aliases use the same routes, exports, agent tools and <GenieChat alias> as chat spaces. The constructor rejects an alias that appears in both maps, and an agent with a missing id.
Sending a message: the agent stream is mapped onto the existing events:
- response.created → message_start
- each reasoning step → a new reasoning event
- execute_sql / generate_visualization → status (EXECUTING_QUERY / GENERATING_VISUALIZATION)
- on response.completed, the message is read back for its query and chart attachments, and the answer text is added as a text attachment, which is where chat mode carries its answer. Then message_result and query_result follow.
- a failed, truncated or externally cancelled stream yields error.
- the turn honours the plugin's timeout (0 = no limit); a timeout yields error.
- a turn that is aborted (the client left past the stream's reconnect grace window) or times out after response.created is cancelled upstream with genieCancelResponse.
History and polling: for agent aliases, GET /:alias/conversations/:id and the pending-message poll fill in the answer text and reasoning from the items endpoint. If that call fails, they return the messages without the fill-in instead of failing.
Charts: a new route, GET /api/genie/:alias/conversations/:cid/messages/:mid/attachments/:aid/visualization, runs on behalf of the user and streams the PNG with image/png, nosniff and Cache-Control: private. The ids are checked against ^[\w-]+$ before they reach the SDK, because the SDK doesn't encode the resource name.
UI:
- useGenieChat appends reasoning events to the in-flight answer and resolves each viz attachment to its image URL (visualizationUrls).
- GenieChatMessage shows a collapsible "Thinking" section, which stays open until the answer arrives.
- For a query with a viz attachment, GenieQueryVisualization (new optional chartImageUrl prop) shows Genie's chart as a lazy-loaded <img> in its Chart tab, in place of the inferred chart. The Table tab is unchanged, and if the image fails to load the inferred chart comes back.
Playground and docs: the dev playground gets a Chat / Agent switch, and the genie docs get an "Agent mode" section and the visualization endpoint.

Charts for chat spaces (second commit)

The chat API also returns viz attachments when Genie draws a chart. This commit maps them in chat mode too, so chat spaces and agents behave the same way for queries with a chart. Chat answers without a chart are unchanged.

API changes (all additive)

IGenieConfig.agents
the reasoning member of GenieStreamEvent
GenieAttachmentResponse.viz
GenieMessageResponse.reasoning
GenieMessageItem.reasoning and GenieMessageItem.visualizationUrls
GenieQueryVisualization's chartImageUrl / chartImageAlt props
the visualization route

Tests

Connector (connectors/genie/tests/agent.test.ts), using fixtures trimmed from a real agent turn:
- the event mapping and its order, including the answer text, reasoning and viz attachments
- the request body for new and follow-up turns
- failed and truncated streams, and agent error bodies
- history, poll and getConversation fill-in across item pages
- downloadVisualization, including rejected ids
Connector (client.test.ts): chat-mode viz mapping for both live messages and history.
Plugin (genie.test.ts):
- agents validation and tools
- agent vs chat routing
- history through the items endpoint
- the visualization route: the piped PNG and its headers, a 400 on bad ids, a 404 on an unknown alias, and Genie's 404 passed through
UI: a useGenieChat test for reasoning and image URLs, and a GenieChatMessage test that the PNG replaces the inferred chart, the inferred chart stays when there's no viz, and reasoning renders.
Full suite: pnpm build, pnpm docs:build, pnpm check, pnpm -r typecheck and pnpm test all pass (6162 passed, 1 skipped).

End-to-end check

I ran the dev playground locally against a Databricks workspace, with one Genie space on sample data configured both as a chat alias and as an agent alias:

Agent mode, chart question: the thinking steps and status show while the agent works. Then the answer text arrives, with the SQL card and Genie's PNG chart.
Reload with ?conversationId=: the history shows the answer text and the chart.
Follow-up: it stays in the same conversation and renders a second chart.
Chat mode: a chart question shows Genie's PNG in the Chart tab, and the Table tab still has the rows (the same in agent mode). A question without a chart renders exactly as before (text, SQL card, inferred table).
Visualization route with curl: it returns a real PNG with the headers above. A path-traversal id gets a 400, and a query attachment id gets a 404.
Leaving mid-turn: closing the browser, or navigating to another page of the app, during an agent turn ends the message as CANCELLED about 15 s later (the stream's reconnect grace window).

Screenshots

Agent mode, while the agent works:

Agent mode, the answer (thinking collapsed):

Agent mode, Genie's chart in the Chart tab under the SQL card:

Chat mode, Genie's chart in the Chart tab:

Chat mode, the same answer's Table tab:

Chat mode, an answer without a chart, unchanged:

Known limitations

The agent's first ("preamble") reasoning step appears in the live stream but not in the items endpoint, so a reloaded conversation can show fewer thinking steps than the live answer did.
Chat-mode viz attachments have no title, so the image's alt text falls back to "Visualization".

Closes #729

This pull request and its description were written by Isaac.

The genie plugin only used the Genie chat API, which returns no agentic reasoning and no rendered charts. Genie agent mode does, through a separate API. Spaces registered under the new `agents` config map use agent mode on the same routes, exports and agent tools as chat spaces: - Sending a message calls POST /api/2.0/genie/agents/{id}/responses with visualizations enabled and maps its stream onto the existing events: message_start, status (per tool call), a new `reasoning` event per reasoning step, then message_result and query_result. The completed message is read back for its query and chart attachments; the answer text, which that message doesn't carry, is added as a text attachment. - History and pending-message polling add the answer text and reasoning from the conversation's items endpoint. - A new visualization route streams the PNG Genie rendered for a `viz` attachment, so the UI can show it in an <img>. GenieChat shows the reasoning while the agent works and shows Genie's chart in place of the inferred chart for queries that have one. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Fabian Jakobs <fabian.jakobs@databricks.com>

The Genie chat API also returns `viz` attachments when Genie draws a chart, and the genie plugin dropped them. Map them in chat mode as well, so chat spaces and agents behave the same: the UI loads Genie's PNG through the visualization route and shows it in place of the inferred chart for queries that have one. Answers without a chart are unchanged. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Fabian Jakobs <fabian.jakobs@databricks.com>

github-actions · 2026-11-12T17:54:54Z

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

`@databricks/appkit`

npm tarball (packed): 1.3 MB (+10.0 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	1.3 MB (+11 KB)	482 KB (+3.4 KB)
Type declarations	471 KB (+1.1 KB)	170 KB (+438 B)
Source maps	2.6 MB (+23 KB)	896 KB (+6.4 KB)
Other	11 KB	3.7 KB
Total	4.4 MB (+35 KB)	1.5 MB (+10 KB)

Per-entry composition (own code — deps external (as shipped))

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
`.`	110 KB (+2.2 KB)	2.5 KB (+1 B)	112 KB (+2.2 KB)	external	355 KB (+6.9 KB)
`./beta`	113 KB (-3 B)	486 B	114 KB (-3 B)	external	339 KB
`./testing`	46 KB	33 KB (-3 B)	79 KB (-3 B)	external	229 KB
`./tsdown`	520 B	0 B	520 B	external	813 B
`./type-generator`	26 KB	0 B	26 KB	external	74 KB

Chunks:

Entry	Chunk	Load	Size (gz)
`.`	`index.js`	initial	105 KB
`.`	`utils.js`	initial	4.6 KB
`.`	`remote-tunnel-manager.js`	lazy	2.5 KB
`./beta`	`beta.js`	initial	93 KB
`./beta`	`modular.js`	initial	6.0 KB
`./beta`	`stream-manager.js`	initial	5.9 KB
`./beta`	`service-context.js`	initial	4.3 KB
`./beta`	`databricks.js`	initial	3.3 KB
`./beta`	`client.js`	initial	648 B
`./beta`	`index.js`	initial	20 B
`./beta`	`supervisor-api.js`	lazy	192 B
`./beta`	`databricks.js`	lazy	179 B
`./beta`	`index.js`	lazy	115 B
`./testing`	`manifest.js`	initial	33 KB
`./testing`	`index.js`	initial	10 KB
`./testing`	`wide-event-emitter.js`	initial	2.9 KB
`./testing`	`index.js`	lazy	29 KB
`./testing`	`remote-tunnel-manager.js`	lazy	2.5 KB
`./testing`	`utils.js`	lazy	1.8 KB
`./tsdown`	`index.js`	initial	520 B
`./type-generator`	`index.js`	initial	26 KB

`@databricks/appkit-ui`

npm tarball (packed): 409 KB (+3.0 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	440 KB (+3.6 KB)	151 KB (+963 B)
Type declarations	272 KB (+1.0 KB)	100 KB (+390 B)
Source maps	869 KB (+7.0 KB)	293 KB (+2.0 KB)
CSS	16 KB	3.2 KB
Total	1.6 MB (+12 KB)	547 KB (+3.3 KB)

Per-entry composition (consumer bundle — deps bundled, peerDeps external)

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
`./js`	5.3 KB	49 KB	55 KB	208 KB	14 KB
`./js/beta`	2.1 KB	0 B	2.1 KB	0 B	5.0 KB
`./react`	441 KB (+592 B)	49 KB	489 KB (+592 B)	1.4 MB (+1 B)	179 KB (+1.7 KB)
`./react/beta`	14 KB	0 B	14 KB	25 KB	16 KB

Chunks:

Entry	Chunk	Load	Size (gz)
`./js`	`index.js`	initial	5.2 KB
`./js`	`chunk`	initial	120 B
`./js`	`apache-arrow`	lazy	49 KB
`./js/beta`	`beta.js`	initial	2.1 KB
`./react`	`index.js`	initial	439 KB
`./react`	`tslib`	initial	2.1 KB
`./react`	`apache-arrow`	lazy	49 KB
`./react/beta`	`beta.js`	initial	14 KB

github-actions · 2026-11-12T18:01:42Z

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 43399389548 -R databricks/appkit -n appkit-template-0.94.1-pr.1359df3-fjakobs-genie-agent-mode-732 -D appkit-pr-732 \
  && unzip -o "appkit-pr-732/appkit-template-0.94.1-pr.1359df3-fjakobs-genie-agent-mode-732.zip" -d "appkit-pr-732" \
  && databricks apps init --template "appkit-pr-732"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

calvarjorge

Nice work. I replayed a few agent and chat turns on a staging workspace, and the event mapping, the item order and paging, the viz/query attachment ids, the PNG download and the agent error body all match what this PR expects. Two things to fix before merge, inline below: the result table disappears when Genie draws a chart, and agent turns have no timeout. The other three comments are smaller.

For a query with a Genie chart, GenieChatMessage replaced the whole GenieQueryVisualization with the PNG, so the Table tab and its rows were gone for every chart answer, in chat mode too. GenieQueryVisualization now takes an optional chartImageUrl: the Chart tab shows that image and the Table tab is unchanged. If the image fails to load, it falls back to the inferred chart. Charts whose query result didn't load are still shown as plain images. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Fabian Jakobs <fabian.jakobs@databricks.com>

…tems errors Review follow-ups for Genie agent mode: - Agent turns honour the plugin's `timeout` (0 = no limit). The stream's timeout interceptor only covered creating the generator, so a stalled turn stayed open indefinitely. A timeout now yields an `error` event; only a caller abort ends the stream silently. - A turn that is aborted (the client left past the reconnect grace window) or times out after `response.created` is cancelled upstream with genieCancelResponse, so the agent stops working for nobody. - A turn cancelled elsewhere ends with `response.completed` and status "cancelled"; it now yields an error instead of an empty answer. - If the agent items call fails, history and polling return the messages without the answer text instead of failing. That also keeps chat conversations loadable after their alias moves from `spaces` to `agents`, since the items endpoint rejects chat conversations. - The visualization route uses pipeline(), which also cancels the upstream download when the client goes away. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Fabian Jakobs <fabian.jakobs@databricks.com>

calvarjorge

Thanks for the quick and thorough fixes. I re-checked all five at 55460b4, and build, typecheck, lint and the genie/stream tests pass locally. Approving, with one small follow-up on the items fallback in its thread.

fjakobs and others added 2 commits November 12, 2026 17:22

fjakobs requested a review from a team as a code owner November 12, 2026 17:51

fjakobs requested a review from ditadi November 12, 2026 17:51

fjakobs requested a review from MarioCadenas November 12, 2026 18:02

calvarjorge reviewed Nov 12, 2026

View reviewed changes

fjakobs and others added 2 commits November 13, 2026 09:11

calvarjorge approved these changes Nov 13, 2026

View reviewed changes

fjakobs merged commit 2248be0 into main Nov 13, 2026
14 checks passed

fjakobs deleted the fjakobs/genie-agent-mode branch November 13, 2026 11:07

fjakobs mentioned this pull request Nov 13, 2026

fix(appkit-ui): describe chat-mode Genie charts by their query #736

Open

Navigation Menu

Repository navigation

feat(appkit): support Genie agent mode with reasoning and visualizations - #732

feat(appkit): support Genie agent mode with reasoning and visualizations#732
fjakobs merged 4 commits into
mainfrom
fjakobs/genie-agent-mode

fjakobs commented Nov 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

calvarjorge left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

calvarjorge left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

fjakobs commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

What the agent API returns

Fix

Agent mode (first commit)

Charts for chat spaces (second commit)

API changes (all additive)

Tests

End-to-end check

Screenshots

Known limitations

Uh oh!

github-actions Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Bundle size report

@databricks/appkit

@databricks/appkit-ui

Uh oh!

github-actions Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 AppKit PR bot

🔬 Run evals

📦 Try this PR's app template

Uh oh!

calvarjorge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

calvarjorge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fjakobs commented Nov 12, 2026 •

edited

Loading

github-actions Bot commented Nov 12, 2026 •

edited

Loading

`@databricks/appkit`

`@databricks/appkit-ui`

github-actions Bot commented Nov 12, 2026 •

edited

Loading