Repository navigation
Conversation
Until now the install hint was printed only to Claude Code. The nine other registry agents (Codex, Cursor, GitHub Copilot CLI, OpenCode, Gemini CLI, Antigravity, Pi, Goose, Kiro) now also get a one-line hint, but only when that agent's own Databricks tooling is missing: Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents <agent> --scope global - The throttle lasts one hour (hintSessionTTL). Keys are "<agent>:session:<id>" (Codex: CODEX_THREAD_ID) or "<agent>:machine" for agents without a session id, so a Claude marker never silences Codex when both report the same id. - If the user cache can't be written (read-only sandbox, or DATABRICKS_CACHE_ENABLED=false), non-Claude agents get no hint at all. - Agents the CLI can't install for (Windsurf, VS Code agent, Amp, ...) and nested agents get no hint. - installer.globalPluginRecorded is now exported as GlobalPluginRecorded, so cmd/root can pass it to MaybeHint. - The telemetry README and the AitoolsHintEvent doc now describe the event for every agent; the new line logs AGENT_INSTALL_TEXT. Co-authored-by: Isaac <no-reply@databricks.com>
f49b71b to
735415e
Compare
|
Commit: Regression test reportSelected 4 changed tests (limit=20, 0 not selected) 4 tests: regression=4 unreleased=0 coverage=0 failing=0
TestAccept/air/image-push: branch ✅ | main (eeb4ef9) ❌ | latest (v1.26.0) ❌main (eeb4ef9): latest (v1.26.0): same output as main (eeb4ef9) TestAccept/cmd/aitools/agent-hints: branch ✅ | main (eeb4ef9) ❌ | latest (v1.26.0) ❌main (eeb4ef9): latest (v1.26.0): same output as main (eeb4ef9) TestAccept/cmd/aitools/claude-config-dir: branch ✅ | main (eeb4ef9) ❌ | latest (v1.26.0) ❌main (eeb4ef9): latest (v1.26.0): same output as main (eeb4ef9) TestAccept/cmd/aitools/telemetry/hint: branch ✅ | main (eeb4ef9) ❌ | latest (v1.26.0) ❌main (eeb4ef9): latest (v1.26.0): |
✅ Integration test reportCommit: 735415e
Top 6 slowest tests (at least 2 minutes):
|
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 735415e. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 9 non-blocking comment(s) and 2 question(s) below.
This PR extends the stderr install hint from Claude Code to the nine other registry agents. Each agent is hinted only when its own Databricks tooling is missing. The hint is throttled per Codex thread or per agent per machine, and prints only when the throttle marker persisted. The implementation and tests look solid, CI is green, and nothing is blocking. Two things are worth fixing: the version --check tip still assumes only Claude Code gets the root hint, and the tooling probe now runs twice per command. The hint can also be wrong or unhelpful in two cases: Codex/Copilot without their CLI on PATH, and Cursor's own plugin.
Questions
- Claude Code's throttle marker stays best-effort, while the other agents print only after their marker is persisted (
durableThrottle). So with an unwritable cache (a sandbox, a read-only home), Claude Code still prints its hint lines on every command. That is the casedurableThrottleprevents for the others. Is the difference intended, for example so Claude Code users in a sandbox still see the hint? Or should all agents use the durable claim? If it's intended, could thedurableThrottlecomment say why? (non-blocking) - The hint relies only on each agent's SDK env marker. For Claude Code, the code notes that
CLAUDECODEalso leaks into IDE terminals people type in, and the native line has an extra agent-only gate. For the IDE-first agents (KIRO,ANTIGRAVITY_AGENT), has anyone checked that the marker is set only for commands the agent runs, and not in the integrated terminal or the extension host? If it isn't, a person (or the Databricks IDE extension) running the CLI there would get the hint on stderr once an hour. (non-blocking)
| nativeKind: protos.AitoolsHintKindClaudeCodeNative, | ||
| } | ||
| default: | ||
| if a := AgentForProvider(provider); a != nil { |
There was a problem hiding this comment.
[nice to have] MaybeVersionCheckTip (libs/aitools/agents/recommend.go:33) still skips only Claude Code, on the grounds that "MaybeHint already covers them". This PR makes MaybeHint cover nine more agents but leaves that skip unchanged. So with CODEX_CI=1 (or CURSOR_AGENT=1, …), no Databricks tooling and ~/.codex present, databricks version --check prints the new … install --agents codex --scope global line from the root hook. It then prints Tip: … Run 'databricks aitools install'. That is two different commands in one run, and the tip isn't throttled. Skip the tip whenever the calling agent gets a root hint (hintForAgent(...) != nil, or CallingAgent() != nil), update that comment, and add a non-Claude case next to the Claude one in acceptance/cmd/version/script.
| // Recommend installing Databricks AI tooling to the calling coding agent | ||
| // when it drives the CLI without the tooling installed (best-effort: a | ||
| // stderr line and a telemetry event that is uploaded at exit). | ||
| agents.MaybeHint(ctx, cmd, installer.GlobalPluginRecorded) |
There was a problem hiding this comment.
[nice to have] installer.WithCallingAgentToolingInUserAgent (root.go:88) already runs a.DatabricksTooling(ctx, GlobalPluginRecorded) for the same calling agent, and MaybeHint then runs it again (hint_agents.go:35-38). Users who have the tooling never get a throttle marker. So under these nine agents, every one of their commands reads the skills dir, walks the project dirs up to the repo root and reads the install state twice. The "check the throttle before the filesystem probe" ordering in MaybeHint doesn't save anything either, because the first probe has already run. Compute the Tooling once in PersistentPreRunE and pass it to both (e.g. MaybeHint(ctx, cmd, tooling)). That also makes PluginRecordedFunc and the GlobalPluginRecorded export unnecessary.
| // installed with the agent's own CLI rather than `databricks aitools install` | ||
| // leaves no trace there for those two, so their users are hinted; the | ||
| // throttle bounds that to once per session or per TTL. | ||
| func agentInstallHint(a *Agent, pluginRecorded PluginRecordedFunc) *agentHint { |
There was a problem hiding this comment.
[nice to have] For Codex and Copilot, --agents <name> without --skills-only takes the plugin path. When codex/copilot isn't on PATH (for example an IDE extension or desktop app that bundles its own binary), that path returns cli-not-on-path and exits non-zero, with no fallback to skills. Nothing gets recorded, so each new Codex thread is told again to "first run" the same failing command. The description lists this as a known risk. A cheap mitigation: when the hint is about to print, check exec.LookPath(a.Binary), and if it fails, hint --skills-only instead. That writes to the agent's own skills dir, so the hint stops afterwards.
| // The hint is gated on the agent's own tooling (Agent.DatabricksTooling, with | ||
| // pluginRecorded as the plugin source for Codex and Copilot). A plugin | ||
| // installed with the agent's own CLI rather than `databricks aitools install` | ||
| // leaves no trace there for those two, so their users are hinted; the | ||
| // throttle bounds that to once per session or per TTL. |
There was a problem hiding this comment.
[nice to have] The registry notes that Cursor has a Databricks plugin, which users install from Cursor itself because the CLI can't install it headlessly. Cursor has neither Plugin nor a manifest reader, so DatabricksTooling only looks at ~/.cursor/skills. A Cursor user who has that plugin (and no raw skills) is told "Databricks skills are not installed" every hour, and following the hint adds raw skills next to the plugin. OpenCode has a similar gap for skills it loads from Claude-compatible dirs such as ~/.claude/skills. Either detect where the Cursor plugin is installed, or at least name Cursor here and in the Merge Danger section, next to Codex and Copilot.
| recorded := func() bool { return pluginRecorded(ctx, a.Name) } | ||
| return a.DatabricksTooling(ctx, recorded) != ToolingNone | ||
| }, | ||
| text: "Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents " + a.Name + " --scope global", |
There was a problem hiding this comment.
[nit] This literal repeats hintText, whose comment says the wording should change only after thorough testing. The next wording change to hintText would leave the other nine agents on the old text. .agents/rules/style-guide-go.md asks to build on an existing constant in cases like this: hintText + " --agents " + a.Name + " --scope global".
| // sessionKeyPrefix namespaces the session id in the throttle key, so one | ||
| // agent's marker never silences another agent that reuses the same id. | ||
| // Empty for Claude Code, whose key is the raw session id. | ||
| sessionKeyPrefix string | ||
|
|
||
| // fallbackKey keys the throttle when there is no session id, so the hint | ||
| // prints at most once per hintSessionTTL on this machine. Empty means no | ||
| // throttle without a session id (Claude Code). | ||
| fallbackKey string | ||
|
|
||
| // durableThrottle prints the text only once its throttle marker is | ||
| // persisted (see claimHintMarker), so an unwritable cache can't turn the | ||
| // hint into a line on every command. False for | ||
| // Claude Code, whose marker write stays best-effort. | ||
| durableThrottle bool |
There was a problem hiding this comment.
[nit] The throttleKey comment (hint.go:305) says Claude's key "is the raw id and must not change". But cache.NewCache stores markers under a per-CLI-version directory, and nothing else reads this bucket, so prefixing Claude's key would break nothing. With the same <agent>:session:<id> key for Claude, throttleKey could derive the prefix from the agent name, and sessionKeyPrefix could go. At minimum, the comment should drop that reason. The durableThrottle comment also has a stray line break ("False for" / "Claude Code").
| // agentInstallHint returns the hint for a non-Claude registry agent. The text | ||
| // keeps Claude Code's imperative wording (hintText) and names the agent and | ||
| // scope, so the install acts on the calling agent and never stops at the agent | ||
| // picker or the scope prompt when the agent's shell is a terminal. |
There was a problem hiding this comment.
[nit] Agent callers never see the agent picker or the scope prompt anyway: installPromptsSupported (cmd/aitools/install.go:41-43) turns prompts off whenever AgentProvider() is set, and the non-interactive scope defaults to global. What --agents <name> actually does is target the calling agent, even when it isn't detected on the machine; --scope global only makes the default explicit. Could the comment say that instead?
| // in the shells it spawns. Only verified variables are listed; an agent missing | ||
| // here is throttled per machine instead (see agentInstallHint). | ||
| var agentSessionEnv = map[string]string{ | ||
| // Set by codex-cli (0.166.0) for every shell tool call, next to CODEX_CI. |
There was a problem hiding this comment.
[nit] .agents/rules/comments.md asks for a source reference URL when detecting environment variables. This comment names the codex-cli version but not where the variable is set. Please link the Codex source that exports CODEX_THREAD_ID, as hint.go does for the Claude hint format.
| assert.Equal(t, agentInstallHint(ByName(NameCodex), noPluginRecorded).text+"\n", stderr.String()) | ||
| } | ||
|
|
||
| func TestMaybeHint_UnwritableCacheSkipsHint(t *testing.T) { |
There was a problem hiding this comment.
[nit] This chmod-based test is skipped on Windows and when running as root, so on Windows nothing covers the "no persisted marker, no hint" rule. A second case with DATABRICKS_CACHE_ENABLED=false, which the claimHintMarker comment and the description both mention, reaches the same branch on every OS without chmod.
Stacked on #7585.
Summary
Until now the install hint was printed only to Claude Code. With this PR, the nine other registry agents (Codex, Cursor, GitHub Copilot CLI, OpenCode, Gemini CLI, Antigravity, Pi, Goose, Kiro) also get a one-line hint, but only when that agent's own Databricks tooling is missing:
MaybeHint(ctx, cmd, pluginRecorded) # cmd/root passes installer.GlobalPluginRecorded hintForAgent(provider) claude-code -> existing Claude hint + registry agent -> agentInstallHint(agent) + installed = Agent.DatabricksTooling (install record is the plugin source for Codex/Copilot) skip aitools commands, auth token, CLI-spawned processes, no output stream - throttle per Claude session id + throttle key = "<agent>:session:<id>" (Codex: CODEX_THREAD_ID) + | "<agent>:machine" (agents without a session id) if installed: return + claim marker; non-Claude agents print only if the marker persisted print line(s), log aitools_hint_event (AGENT_INSTALL_TEXT for the new line)hintSessionTTL). Session keys carry the agent name as a prefix, so a Claude marker never silences Codex when both report the same id.DATABRICKS_CACHE_ENABLED=false), non-Claude agents get no hint at all. Otherwise they would get it on every command.installer.globalPluginRecordedis now exported asGlobalPluginRecorded, socmd/rootcan pass it in.AitoolsHintEventdoc now describe the event for every agent..nextchanges/cli/aitools-hint-other-agents.md.Evidence
With
hintForAgent's new registry-agent branch temporarily removed (tests unchanged), then restored, at this head:ok libs/aitools/agents(48 tests and subtests pass) andok acceptanceforTestAccept/cmd/aitools/agent-hints.agent-hintsgolden shows a Codex hint once per thread, Cursor once per machine, Copilot mapped tocopilot, no hint for Gemini when its skills are installed, and no hint for Windsurf, the VS Code agent or nested agents.task checksall pass.go vetpasses withGOOS=windowsandGOOS=darwin.Merge Danger
Door: two-way
The hint is stderr-only and best-effort. It never changes exit codes or stdout, and it does no network calls. Reverting this PR removes it.
Blast Radius: agent-sessions
aitools_hint_event, which uploads at exit only if the command resolved auth.databricks aitools install, isn't detected, so those users keep getting the hint once per throttle window.--agents <name> --scope globalfails if the agent's own CLI isn't on PATH.go vet. On Windows, the read-only-cache test is skipped.This pull request and its description were written by Isaac.