Repository navigation
aitools: report a Windows shim that printed help but exited non-zero under its own reason #7476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * `databricks aitools install` and `databricks aitools update` now report a coding agent CLI shim (`.cmd` or `.bat`) on Windows that prints its plugin help and then exits with an error as `shim-exited-with-error`, with the command to run to see the error, instead of reporting the CLI as not on PATH. ([#7476](https://github.com/databricks/cli/pull/7476)) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,7 +6,10 @@ import ( | |
| "fmt" | ||
| "maps" | ||
| "os/exec" | ||
| "path/filepath" | ||
| "regexp" | ||
| "strings" | ||
| "sync" | ||
| "time" | ||
|
|
||
| "github.com/databricks/cli/libs/aitools/agents" | ||
|
|
@@ -52,6 +55,10 @@ const ( | |
| // ReasonCLINotOnPath: the agent's CLI binary is not on PATH, or its CLI does | ||
| // not expose a working `plugin` subcommand. | ||
| ReasonCLINotOnPath = "cli-not-on-path" | ||
| // ReasonShimExitedWithError: the agent's Windows .cmd/.bat shim printed its | ||
| // `plugin --help` but exited with an error. The install stops here rather | ||
| // than run plugin commands through a shim that may fail part way. | ||
| ReasonShimExitedWithError = "shim-exited-with-error" | ||
| // ReasonInstallFailed: the agent's plugin CLI ran but returned an error. | ||
| ReasonInstallFailed = "install-failed" | ||
| // ReasonNoPlugin: the agent has no installable plugin. Callers filter these | ||
|
|
@@ -308,11 +315,68 @@ func probePluginCLI(ctx context.Context, agent *agents.Agent) (string, error) { | |
| return "", &BlockedError{Agent: agent.Name, Reason: ReasonCLINotOnPath, Detail: err.Error()} | ||
| } | ||
| if _, err := runAgentCmd(ctx, pluginProbeTimeout, []string{bin, "plugin", "--help"}); err != nil { | ||
| if perr, ok := windowsShimPrintedHelp(bin, err); ok { | ||
| return "", &BlockedError{Agent: agent.Name, Reason: ReasonShimExitedWithError, Detail: shimExitedDetail(bin, perr)} | ||
| } | ||
| return "", newBlockedError(agent, ReasonCLINotOnPath, err) | ||
| } | ||
| return bin, nil | ||
| } | ||
|
|
||
| // shimExitedDetail explains a shim that printed its plugin help but exited with | ||
| // an error. It keeps the shim's stderr, since a re-run may show only the help, | ||
| // and names how to read the exit code, which neither Windows shell prints. The | ||
| // path stays absolute and keeps its backslashes so it can be pasted into | ||
| // Command Prompt or PowerShell as is, and it is quoted because a profile path | ||
| // can contain spaces. PowerShell needs the & call operator before a quoted path. | ||
| // Command Prompt expands %ERRORLEVEL% when it reads a line, so the hint reads it | ||
| // in a separate command after the shim has run. | ||
| func shimExitedDetail(bin string, perr *process.ProcessError) string { | ||
| detail := fmt.Sprintf("printed its plugin help but exited with an error (%s)", perr.Err) | ||
| if stderr := strings.TrimSpace(perr.Stderr); stderr != "" { | ||
| detail += ": " + stderr | ||
| } | ||
| return detail + fmt.Sprintf("; to see the error and exit code, run `& \"%[1]s\" plugin --help; $LASTEXITCODE` in PowerShell or `\"%[1]s\" plugin --help` and then `echo %%ERRORLEVEL%%` in Command Prompt, and re-run this command once it exits with 0", bin) | ||
| } | ||
|
|
||
| // pluginHelpUsage, pluginHelpInstall and pluginHelpMarket match the help that | ||
| // `<agent> plugin --help` prints: a "Usage: <bin> plugin" line and an indented | ||
| // command list that includes install (add for Codex) and marketplace. An error | ||
| // message that merely mentions those words has neither shape. | ||
| var ( | ||
| pluginHelpUsage = sync.OnceValue(func() *regexp.Regexp { return regexp.MustCompile(`(?im)^usage:[ \t]+\S+[ \t]+plugin\b`) }) | ||
| pluginHelpInstall = sync.OnceValue(func() *regexp.Regexp { return regexp.MustCompile(`(?m)^[ \t]+(install|add)\b`) }) | ||
| pluginHelpMarket = sync.OnceValue(func() *regexp.Regexp { return regexp.MustCompile(`(?m)^[ \t]+marketplace\b`) }) | ||
| ) | ||
|
|
||
| // windowsShimPrintedHelp reports whether a failed probe is a Windows shim that | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [nice to have]
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Partly fixed in 4f48110. There is no public upstream report of this exit code that I can link, so the |
||
| // printed the full `plugin --help` and then exited with an error, and returns | ||
| // the failed process. Such a CLI is present, so reporting it as not on PATH | ||
| // would point the user at the wrong fix. Agent binaries are bare names, so | ||
| // LookPath returns a .cmd or .bat path only on Windows. Only stdout is matched: | ||
| // the agent CLIs print --help to stdout, and help on stderr usually means the | ||
| // CLI rejected its arguments. | ||
| // | ||
| // TestProbeRealCmdShimPrintedHelp runs a real .cmd that prints the help and | ||
| // exits 1, which checks how this path sees a shim's exit code on Windows. | ||
| func windowsShimPrintedHelp(bin string, err error) (*process.ProcessError, bool) { | ||
| if ext := strings.ToLower(filepath.Ext(bin)); ext != ".cmd" && ext != ".bat" { | ||
| return nil, false | ||
| } | ||
| if _, ok := errors.AsType[*agentCmdTimeoutError](err); ok { | ||
| return nil, false | ||
| } | ||
| perr, ok := errors.AsType[*process.ProcessError](err) | ||
| if !ok { | ||
| return nil, false | ||
| } | ||
| out := perr.Stdout | ||
| if !pluginHelpUsage().MatchString(out) || !pluginHelpInstall().MatchString(out) || !pluginHelpMarket().MatchString(out) { | ||
| return nil, false | ||
| } | ||
| return perr, true | ||
| } | ||
|
|
||
| // InstallPluginForAgent registers the databricks marketplace and installs the | ||
| // plugin through the agent's own CLI, returning the record to persist in state. | ||
| // It never falls back to skills: a blocked install returns a *BlockedError. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| //go:build windows | ||
|
|
||
| package installer | ||
|
|
||
| import ( | ||
| "os" | ||
| "path/filepath" | ||
| "strings" | ||
| "testing" | ||
|
|
||
| "github.com/databricks/cli/libs/aitools/agents" | ||
| "github.com/stretchr/testify/assert" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| // TestProbeRealCmdShimPrintedHelp runs a real .cmd shim that prints the plugin | ||
| // help and exits 1, so the probe sees the exit code and stdout that cmd.exe | ||
| // actually reports rather than a stub. | ||
| func TestProbeRealCmdShimPrintedHelp(t *testing.T) { | ||
| dir := t.TempDir() | ||
| require.NoError(t, os.WriteFile(filepath.Join(dir, "help.txt"), []byte(copilotPluginHelp), 0o644)) | ||
| shim := filepath.Join(dir, "copilot.cmd") | ||
| require.NoError(t, os.WriteFile(shim, []byte("@type \"%~dp0help.txt\"\r\n@exit /b 1\r\n"), 0o755)) | ||
|
|
||
| origLook := lookPath | ||
| t.Cleanup(func() { lookPath = origLook }) | ||
| lookPath = func(string) (string, error) { return shim, nil } | ||
|
|
||
| _, err := InstallPluginForAgent(t.Context(), pluginAgent(agents.NameCopilot, "GitHub Copilot", "copilot"), "", "v0.2.6") | ||
| var be *BlockedError | ||
| require.ErrorAs(t, err, &be) | ||
| assert.Equal(t, ReasonShimExitedWithError, be.Reason) | ||
| assert.True(t, strings.HasPrefix(be.Detail, "printed its plugin help but exited with an error (exit status 1)"), be.Detail) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nice to have]
ReasonShimExitedWithErrorhas no case inblockedErrorCategory(cmd/aitools/categorize.go:184), so it falls through to the default case and returnsAitoolsErrorCategoryUncategorized. A Windows.cmd/.batshim that prints its plugin help and exits 1 was categorized asCLI_NOT_ON_PATHbefore this PR. It is nowUNCATEGORIZEDin telemetry, in the funnel, and in theerror_categoryfield of--output json(cmd/aitools/install.go:699), so it can't be told apart from unknown errors. That also contradicts the Merge Danger note that there are no API changes. Add a case for the new reason, either to an existing category (CLI_NOT_ON_PATHkeeps the old bucket;PLUGIN_INSTALL_FAILEDis closer to the meaning) or to a new value (which needs the lumberjack proto change first). Add a matching row toTestClassifyInstallErrorincmd/aitools/categorize_test.go.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 38d9620:
blockedErrorCategorymapsReasonShimExitedWithErrortoPLUGIN_INSTALL_FAILED, the closest existing category (the CLI is present and its plugin command failed). No new enum value. Added theblocked shim exited with errorrow toTestClassifyInstallError, and the Merge Danger note now says--output jsonreportsPLUGIN_INSTALL_FAILEDinstead ofCLI_NOT_ON_PATHfor this case.