Repository navigation
aitools: keep installing plugins when the skills install fails - #7479
Conversation
7c162ba to
593c4fe
Compare
891a0e3 to
e15f2f9
Compare
✅ Integration test reportCommit: fbf8382
Top 25 slowest tests (at least 2 minutes):
|
593c4fe to
deca41f
Compare
e15f2f9 to
0fb3e5c
Compare
deca41f to
e980d56
Compare
0fb3e5c to
520f504
Compare
e980d56 to
e155c53
Compare
520f504 to
6709c89
Compare
e155c53 to
a791ac7
Compare
6709c89 to
e7d2703
Compare
a791ac7 to
a45b450
Compare
912057b to
5f9958b
Compare
a45b450 to
eb589c2
Compare
5f9958b to
6f53d25
Compare
eb589c2 to
df49877
Compare
6f53d25 to
bced029
Compare
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at bced029. Each finding was verified against the code; simonfaltum may follow up.
Not approving yet: part of the automated review did not complete.
This PR lets databricks aitools install keep installing plugins for plugin agents such as Claude Code when the skills-group install fails, and still returns the skills error as the command's error. The change is small, focused and green in CI. One non-blocking issue: after a skills failure, recording the plugin now advances the shared state.Release, so aitools version and, with a pinned ref, aitools update can report stale skills as up to date. The other comments are small nits: tests for the joined-error paths, a stale test comment, and the changelog wording.
Comments outside the changed lines
- [nit]
cmd/aitools/install_test.go:531: This comment says the skills-group path "returns before appending any per-agent outcome", but this PR removed that early return. The test still passes because a Cursor-only plan has no plugin items. Suggest rewording it, for example: "Cursor is skills-only, so this fails in the skills-group path, which records no per-agent outcome."
| @@ -630,7 +631,7 @@ func executePlan(ctx context.Context, src installer.ManifestSource, plan []agent | |||
| } | |||
| if len(records) > 0 { | |||
| if err := recordPluginInstallsFn(ctx, opts.Scope, records, ref); err != nil { | |||
There was a problem hiding this comment.
[nice to have] Now that the plugin step runs after a skills failure, RecordPluginInstalls (libs/aitools/installer/plugin.go:487) still sets state.Release to the new ref, even though InstallSkillsForAgents returned before it saved state. The skills on disk and in state.Skills stay at the old release. Example: Cursor's skills are at v0.2.5, a v0.2.6 install fails during the skills fetch, and the Claude Code plugin is recorded. The state then says v0.2.6, aitools version reports "up to date" (cmd/aitools/version.go:132), and with a pinned ref (cli-compat pin or DATABRICKS_SKILLS_REF) aitools update stops at explicit && state.Release == latestTag (libs/aitools/installer/update.go:101) and prints "Already up to date." without updating Cursor's skills. Before this PR, the early return meant nothing was written after a skills failure. Suggest leaving Release unchanged when recording plugins after a skills failure (for example a flag or a separate record call), and adding a test that checks the saved state after the mixed failure.
There was a problem hiding this comment.
Fixed. When the skills install failed, executePlan now records the plugins with an empty ref, and RecordPluginInstalls leaves state.Release unchanged for an empty ref (doc comment updated). TestExecutePlanSkillsFailureStillInstallsPlugins now runs the real RecordPluginInstalls against a temp HOME seeded with a v0.2.5 state, and checks that the saved state has the Claude Code plugin and Release still v0.2.5. Without the fix the test gets v0.2.6. A fresh install whose skills step fails gets Release empty, which matches having no skills installed.
Commit: cad6ebe
| ref, err := installer.GetPluginSkillsRef(ctx) | ||
| if err != nil { | ||
| return outcomes, err | ||
| return outcomes, errors.Join(skillsErr, err) |
There was a problem hiding this comment.
[nit] The new errors.Join(skillsErr, err) returns (ref lookup here, state write at L634) have no tests. No test stubs recordPluginInstallsFn to fail, and the new test pins DATABRICKS_SKILLS_REF, so the ref lookup always succeeds. A case where the skills install and recordPluginInstallsFn both fail would pin down the joined message and the category picked for it. classifyInstallError ranks errors by type, not by order, so a typed plugin-side error such as StateError can outrank an untyped skills error.
There was a problem hiding this comment.
Added TestExecutePlanSkillsAndStateWriteFailuresAreJoined: an untyped skills error, and recordPluginInstallsFn returning a StateError. It pins the joined message (skills error first, then the state error), errors.Is for both, and the category STATE_INVALID, so the type ranking you describe is now explicit in a test. I didn't add a separate test for the ref-lookup path: it uses the same errors.Join(skillsErr, err) return, and making GetPluginSkillsRef fail would mean adding a new seam just for this test.
Commit: cad6ebe
There was a problem hiding this comment.
Update after rebasing onto the current #7478: the ref-lookup failure no longer returns an error there (the plugin version is recorded as unknown), so the only joined-error path left is the state write, and TestExecutePlanSkillsAndStateWriteFailuresAreJoined covers it. PR description updated to match.
Commit: 4f38e11
| @@ -0,0 +1 @@ | |||
| * `databricks aitools install` still installs plugins for the other coding agents when installing skill files fails. ([#7479](https://github.com/databricks/cli/pull/7479)) | |||
There was a problem hiding this comment.
[nit] The pr-checklist (.agents/skills/pr-checklist/SKILL.md:75) asks for an imperative verb, or the command in the present tense with "now". "still installs" reads as if the behavior didn't change. Suggest: "* databricks aitools install now installs plugins for the other coding agents even when installing skill files fails. (#7479)"
There was a problem hiding this comment.
Applied your wording, PR link kept: "databricks aitools install now installs plugins for the other coding agents even when installing skill files fails."
Commit: cad6ebe
bced029 to
cad6ebe
Compare
|
Outside-diff nit on Commit: cad6ebe |
942cec5 to
ddd4225
Compare
efae3db to
6aa9729
Compare
ddd4225 to
45b26f8
Compare
f5bf3ea to
ac0065e
Compare
45b26f8 to
a3e42ce
Compare
ac0065e to
2a658e6
Compare
a3e42ce to
cad677d
Compare
2a658e6 to
51b22e3
Compare
cad677d to
2adef03
Compare
rugpanov
left a comment
There was a problem hiding this comment.
cmd/aitools/install.go:761: [nit] This comment says that without the top-level error, the consumer sees "an empty agents array". After this PR, the array can hold the plugin agents next to a top-level skills error. Update this comment, and the similar one in install_test.go:721.
| releaseRef = "" | ||
| } | ||
| if err := recordPluginInstallsFn(ctx, opts.Scope, records, releaseRef); err != nil { | ||
| return outcomes, errors.Join(skillsErr, err) |
There was a problem hiding this comment.
[nice to have] Skill files and plugin records share one state file per scope. When the skills install fails because that state file cannot be read or parsed, the plugin step still runs marketplace add and plugin install. Then this call fails on the same state file. The plugin is installed but not recorded, so update and uninstall do not manage it, and the JSON output reports it as installed. Before this PR, such a run changed nothing. Go on to the plugins only when the skills step failed on a fetch (*installer.FetchError). Add a test that does not stub recordPluginInstallsFn. In the legacy case at line 668 the plugin is also left unrecorded, on purpose, but only a debug log says so. Print a warning there that tells the user to re-run databricks aitools install.
There was a problem hiding this comment.
Agreed, both parts are real gaps. This PR is already approved with the current behavior, so I'd like to handle them in a follow-up instead of changing the gate here:
- Go on to the plugin step only when the skills step failed on a fetch (
*installer.FetchError). That way an unreadable or invalid state file stops the run beforemarketplace add/plugin install. Add a test that runs the realRecordPluginInstalls. - Print a warning in the untracked-skills branch that tells the user to re-run
databricks aitools install, instead of only a debug log.
For now, in the state-file case the command still fails: the returned error joins the skills error and the state error.
| // such installs. A state file that can't be read counts as no: the caller's | ||
| // next state load reports that error itself. | ||
| func HasUntrackedSkills(ctx context.Context, cliScope string) bool { | ||
| if cliScope != ScopeGlobal { |
There was a problem hiding this comment.
[nit] This returns false for project scope, because "Project scope never had such installs". But InstallSkillsForAgents saves the state only after its loop, so a first project install that fails on the third skill leaves two skills on disk with no state. Then executePlan records the plugins, and the new project state has no skills. update treats that scope as plugin-only, and uninstall leaves those skill files. Check the project dir for skills without state too, and say in the doc that an interrupted install also leaves untracked skills.
There was a problem hiding this comment.
Updated the doc comment: it now says the check covers skills on disk with no state file from either an install that predates state tracking or a first install that failed before saving state. It also says only the global scope is checked, so an interrupted first project install isn't detected. I also aligned the comment at the call site in install.go.
I left the project-scope check itself for a follow-up. It changes when plugins get recorded in project scope, and this PR is already approved with the global-only behavior.
Commit: 1eca02f
51b22e3 to
a4c1544
Compare
2adef03 to
b9488bb
Compare
a4c1544 to
c635c52
Compare
b9488bb to
3e7a5e5
Compare
c635c52 to
b09a485
Compare
3e7a5e5 to
916c212
Compare
b09a485 to
2ee6302
Compare
916c212 to
b6d9a96
Compare
2ee6302 to
d105006
Compare
838c303 to
1eca02f
Compare
d105006 to
924919a
Compare
1eca02f to
fbf8382
Compare
|
Fixed both comments. The Commit: 1eca02f |
924919a to
05df0d0
Compare
fbf8382 to
f3ea264
Compare
`databricks aitools install` no longer stops when installing skill files fails. Agents that get a plugin (Claude Code) don't depend on the skill files, so their plugin is still installed. The skills error is still the command's error. The plugin is recorded in the install state without advancing `state.Release`, because the skill files on disk are still at the old release; otherwise `aitools version` and a pinned `aitools update` would report them as up to date. If skills from an install that predates state tracking are on disk with no state file, the plugin is not recorded at all, so `aitools update` and the install's legacy checks still see those skills. ```diff executePlan install skills for skills agents - on error: return error # plugin agents never attempted + on error: remember skillsErr install plugins for plugin agents - on state write error: return err + on state write error: return errors.Join(skillsErr, err) + record plugins; if skillsErr, keep state.Release # skills on disk are still at the old release + if skillsErr and untracked legacy skills on disk: don't create a plugin-only state file + if skillsErr: return skillsErr # stays the top-level error if per-agent errors: return agentErrors ``` Changelog fragment: `.nextchanges/cli/aitools-install-skills-failure-keeps-plugins.md`. Co-authored-by: Isaac <no-reply@databricks.com>
❌ Integration test reportCommit: eee3c42
51 interesting tests: 39 FAIL, 11 RECOVERED, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
Stacked on #7478.
Stacked on: aitools: drop the Proceed? confirm after the agent picker in install
Summary
databricks aitools installno longer stops when installing skill files fails. Agents that get a plugin (Claude Code) don't depend on the skill files, so their plugin is still installed. The skills error is still the command's error. The plugin is recorded in the install state without advancingstate.Release, because the skill files on disk are still at the old release; otherwiseaitools versionand a pinnedaitools updatewould report them as up to date. If the global scope has skills on disk but no state file (an install that predates state tracking, or an interrupted first install), the plugin is not recorded at all, soaitools updateand the install's legacy checks still see those skills.Changelog fragment:
.nextchanges/cli/aitools-install-skills-failure-keeps-plugins.md.Evidence
New test
TestExecutePlanSkillsFailureStillInstallsPlugins(cmd/aitools/install_test.go): plan with Cursor (skills) and Claude Code (plugin), skills install returns an HTTP 503 manifest fetch error. Run withgo test ./cmd/aitools -run TestExecutePlanSkillsFailureStillInstallsPlugins.cmd/aitools/install.goreverted to the parent): fails, no plugin is installed.errors.Is), one plugin install is made, Claude Code's outcome isinstalled, and the saved state records the plugin whileReleasestays at the earlierv0.2.5.New test
TestExecutePlanSkillsAndStateWriteFailuresAreJoined: the skills install and the state write both fail. The returned error joins both (skills first), and its category isSTATE_INVALID, becauseclassifyInstallErrorranks errors by type.New test
TestExecutePlanSkillsFailureKeepsUntrackedSkillsVisible: legacy skills on disk with no state file, and the skills install fails. The plugin is installed, but no state file is created, soHasUntrackedSkills(the signalaitools updateand the install's legacy checks rely on) is still true. Without the guard, a plugin-only state file is written and the test fails.Merge Danger
Door: two-way
Revert the commit to bring the old behavior back. Nothing is migrated. After a skills failure the state file now gets the plugin record (with
Releaseunchanged), where before nothing was written. The exception is a global scope with skills on disk and no state file: there nothing is written, and the plugin is recorded by the next install whose skills step succeeds.Blast Radius: small
This pull request and its description were written by Isaac.