Skip to content

Log AitoolsMcpEvent from aitools mcp and the serve proxy - #7554

Draft
simonfaltum wants to merge 22 commits into
aitools-mcp/09-pickersfrom
aitools-mcp/10-telemetry
Draft

Log AitoolsMcpEvent from aitools mcp and the serve proxy#7554
simonfaltum wants to merge 22 commits into
aitools-mcp/09-pickersfrom
aitools-mcp/10-telemetry

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Nov 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This pull request is part of the stack that adds the hidden databricks aitools mcp command group, on top of #7552. It adds typed AitoolsMcpEvent telemetry for the management commands and for serve's start and sign-in prompts. The telemetry schema is not merged yet, so this pull request stays a draft.

Changes

  • AitoolsMcpEvent, its server result and their enums in libs/telemetry/protos. The MCP error categories join AitoolsErrorCategory; their JSON strings are unchanged.
  • add logs ADD and SIGN_IN; login and logout log SIGN_IN and SIGN_OUT; remove logs REMOVE when it touched a server; serve logs PROXY_STARTED and SIGN_IN_PROMPTED.
  • Each command uploads with the profile it bound, including one picked in the picker, and credentials that never refresh a token. A run that sees a rejected credential uploads nothing.
  • The upload's host metadata lookup logs failures at debug level, so a successful command prints no warning.
  • libs/telemetry gets an upload slot (SetUploadConfig, RefuseUpload, UploadConfig). install is unchanged.
  • Events carry enums and counts only. A service outside system.ai is CUSTOM, and its name is not sent.
  • No changelog fragment: the command group is hidden, so nothing here is user-visible yet.

Why

The MCP commands and the proxy have no telemetry, so their use cannot be measured. Background auth resolution, as install uses, could send a refreshed token or another profile's credential, or upload under a profile the command did not bind.

Decisions to review

  • A failed start uploads nothing. PROFILE_NOT_FOUND and PROFILE_HOST_CHANGED are never sent, since the token must not reach a host the record does not name, and AUTH_FAILED would repeat the rejected request. These failures are undercounted.
  • The upload reads the bound host's metadata, an unauthenticated request. remove may wait for the upload after printing its output.
  • Events upload when the command returns, so a killed serve loses its PROXY_STARTED. The enum names and JSON keys must match the telemetry schema change.
  • A service that needs no sign-in has no 401 handling in the proxy, so a rejected credential there still uploads at exit.

Tests

  • aitools_mcp_test.go pins the JSON keys. mcp_telemetry_test.go covers each service, outcome, profile count and rejected credentials. The touched packages pass with -race.
  • TestMcpRemoveUploadIgnoresEnvironmentToken checks that an upload never uses another workspace's token. TestMcpRemoveUploadSkipsDriftedProfile checks the drift guard, and TestBoundUploadConfigNeverRefreshes the refresh rule.
  • TestUnauthorizedRecordsRejectedCredential covers a 401 from the service, and TestSetUploadConfigLogsNoWarningForFailedHostMetadata the metadata warning.
  • TestUploadRefusedSendsNothing covers the upload slot.
  • Acceptance acceptance/cmd/aitools/telemetry/mcp-add, mcp-remove, mcp-sign-in and mcp-serve print the events each command uploaded.

Not tested: the command-funnel events, which come with the funnel pull request, and a live workspace.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: f20df6b26 | Workflow run

Regression test report

Selected 20 changed tests (limit=20, 0 not selected)

20 tests: regression=20 unreleased=0 coverage=0 failing=0
test branch main (8e70c8b) latest (v1.26.0)
TestAccept/cmd/aitools/mcp/add-agent-not-installed ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-default-config ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-remove-agent-clis ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-text ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-two-agents ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-two-profiles ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-unparsable-config ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/drift-profile-missing ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/hidden ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/list-and-remove ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/list-check ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/login-flips-active ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/repair-and-warnings ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/serve-authenticate-tool ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/serve-binding-errors ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/serve-passthrough ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-add ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-remove ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-serve ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-sign-in ✅ ❌ ❌
TestAccept/cmd/aitools/mcp/add-agent-not-installed: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1386: Diff:
        --- cmd/aitools/mcp/add-agent-not-installed/output.txt
        +++ /tmp/TestAcceptcmdaitoolsmcpadd-agent-not-installed1572277677/002/output.txt
        @@ -2,41 +2,58 @@
         === An agent named with --agents that is not installed is refused; the installed agent still gets the server, and the command exits non-zero
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code,antigravity --output json
        -{
        -  "profile": "logfood",
        -  "host": "[DATABRICKS_URL]",
        -  "scope": "global",
        -  "servers": [
        -    {
        -      "name": "genie_one_mcp-logfood",
        -      "service": "system.ai.genie_one_mcp",
        -      "sign_in_required": false,
        -      "agents": [
        -        {
        -          "name": "claude-code",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.claude.json"
        -        },
        -        {
        -          "name": "antigravity",
        -          "status": "failed",
        -          "error_category": "UNKNOWN_AGENT",
        -          "message": "Antigravity is not installed; install it first or leave it out of --agents"
        -        }
        -      ]
        -    }
        -  ],
        -  "error_category": "TYPE_UNSPECIFIED"
        -}
        +Error: unknown flag: --names
         
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
        +
         === Nothing was created for Antigravity
         
         >>> ls -A home
         .claude
        -.claude.json
        -.databricks
         
         === With no installed agent named, the command is refused before any request, and the text output says what to do
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents antigravity
        -Error: Antigravity is not installed; install it first or leave it out of --agents
        +Error: unknown flag: --names
        +
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
        
    acceptance_test.go:1164: 
        LOG.create-genie_one_mcp-logfood: {
        LOG.create-genie_one_mcp-logfood:   "comment": "Ask questions about your data",
        LOG.create-genie_one_mcp-logfood:   "created_by": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "metastore_id": "7a0ca86d-2d8f-7d6a-7a0c-a86d2d8f7e83",
        LOG.create-genie_one_mcp-logfood:   "name": "mcp-services/system.ai.genie_one_mcp",
        LOG.create-genie_one_mcp-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-genie_one_mcp-logfood: }

latest (v1.26.0):

acceptance_test.go:1386: Diff:
        --- cmd/aitools/mcp/add-agent-not-installed/output.txt
        +++ /tmp/TestAcceptcmdaitoolsmcpadd-agent-not-installed3681918030/002/output.txt
        @@ -2,41 +2,58 @@
         === An agent named with --agents that is not installed is refused; the installed agent still gets the server, and the command exits non-zero
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code,antigravity --output json
        -{
        -  "profile": "logfood",
        -  "host": "[DATABRICKS_URL]",
        -  "scope": "global",
        -  "servers": [
        -    {
        -      "name": "genie_one_mcp-logfood",
        -      "service": "system.ai.genie_one_mcp",
        -      "sign_in_required": false,
        -      "agents": [
        -        {
        -          "name": "claude-code",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.claude.json"
        -        },
        -        {
        -          "name": "antigravity",
        -          "status": "failed",
        -          "error_category": "UNKNOWN_AGENT",
        -          "message": "Antigravity is not installed; install it first or leave it out of --agents"
        -        }
        -      ]
        -    }
        -  ],
        -  "error_category": "TYPE_UNSPECIFIED"
        -}
        +Error: unknown flag: --names
         
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
        +
         === Nothing was created for Antigravity
         
         >>> ls -A home
         .claude
        -.claude.json
        -.databricks
         
         === With no installed agent named, the command is refused before any request, and the text output says what to do
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents antigravity
        -Error: Antigravity is not installed; install it first or leave it out of --agents
        +Error: unknown flag: --names
        +
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
        
    acceptance_test.go:1164: 
        LOG.create-genie_one_mcp-logfood: {
        LOG.create-genie_one_mcp-logfood:   "comment": "Ask questions about your data",
        LOG.create-genie_one_mcp-logfood:   "created_by": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "metastore_id": "7a0ca87c-020c-f50a-7a0c-a87c020cf64d",
        LOG.create-genie_one_mcp-logfood:   "name": "mcp-services/system.ai.genie_one_mcp",
        LOG.create-genie_one_mcp-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-genie_one_mcp-logfood: }
TestAccept/cmd/aitools/mcp/add-default-config: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1386: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1386
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1127
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:672
        	Error:      	Not equal: 
        	            	expected: "\n=== Without DATABRICKS_CONFIG_FILE the server is added for the default config file\n\n>>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code\nProfile logfood ([DATABRICKS_URL])\nAgents (named with --agents): claude-code\n\ngenie_one_mcp-logfood  system.ai.genie_one_mcp\n  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json\n\n=== Claude Code config has no pinned variables for the server\n\n>>> gron.py home/.claude.json\njson.mcpServers.genie_one_mcp-logfood.env = {};\n\n=== State holds the binding with no config file\n\n>>> jq .mcp.servers[] | {name, profile, config_file} home/.databricks/aitools/skills/.state.json\n{\n  \"name\": \"genie_one_mcp-logfood\",\n  \"profile\": \"logfood\",\n  \"config_file\": \"\"\n}\n"
        	            	actual  : "\n=== Without DATABRICKS_CONFIG_FILE the server is added for the default config file\n\n>>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code\nError: unknown flag: --names\n\nUsage:\n  databricks aitools [command]\n\nAvailable Commands:\n  install     Install Databricks skills and plugins for coding agents\n  list        List installed skills and plugins\n  uninstall   Uninstall Databricks skills and plugins\n  update      Update installed skills and plugins\n  version     Show installed skills and plugins version\n\nFlags:\n  -h, --help   help for aitools\n\nGlobal Flags:\n      --debug            enable debug logging\n  -o, --output type      output type: text or json (default text)\n  -p, --profile string   ~/.databrickscfg profile\n  -t, --target string    bundle target to use (if applicable)\n\nUse \"databricks aitools [command] --help\" for more information about a command.\n\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,21 +4,27 @@
        	            	 >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code
        	            	-Profile logfood ([DATABRICKS_URL])
        	            	-Agents (named with --agents): claude-code
        	            	+Error: unknown flag: --names
        	            	 
        	            	-genie_one_mcp-logfood  system.ai.genie_one_mcp
        	            	-  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json
        	            	+Usage:
        	            	+  databricks aitools [command]
        	            	 
        	            	-=== Claude Code config has no pinned variables for the server
        	            	+Available Commands:
        	            	+  install     Install Databricks skills and plugins for coding agents
        	            	+  list        List installed skills and plugins
        	            	+  uninstall   Uninstall Databricks skills and plugins
        	            	+  update      Update installed skills and plugins
        	            	+  version     Show installed skills and plugins version
        	            	 
        	            	->>> gron.py home/.claude.json
        	            	-json.mcpServers.genie_one_mcp-logfood.env = {};
        	            	+Flags:
        	            	+  -h, --help   help for aitools
        	            	 
        	            	-=== State holds the binding with no config file
        	            	+Global Flags:
        	            	+      --debug            enable debug logging
        	            	+  -o, --output type      output type: text or json (default text)
        	            	+  -p, --profile string   ~/.databrickscfg profile
        	            	+  -t, --target string    bundle target to use (if applicable)
        	            	 
        	            	->>> jq .mcp.servers[] | {name, profile, config_file} home/.databricks/aitools/skills/.state.json
        	            	-{
        	            	-  "name": "genie_one_mcp-logfood",
        	            	-  "profile": "logfood",
        	            	-  "config_file": ""
        	            	-}
        	            	+Use "databricks aitools [command] --help" for more information about a command.
        	            	 
        	            	+
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/mcp/add-default-config
        	Messages:   	cmd/aitools/mcp/add-default-config/output.txt vs /tmp/TestAcceptcmdaitoolsmcpadd-default-config990958839/002/output.txt
    acceptance_test.go:1164: 
        LOG.create-genie_one_mcp-logfood: {
        LOG.create-genie_one_mcp-logfood:   "comment": "Ask questions about your data",
        LOG.create-genie_one_mcp-logfood:   "created_by": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "metastore_id": "7a0ca874-940d-537c-7a0c-a874940d549b",
        LOG.create-genie_one_mcp-logfood:   "name": "mcp-services/system.ai.genie_one_mcp",
        LOG.create-genie_one_mcp-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-genie_one_mcp-logfood: }

latest (v1.26.0):

acceptance_test.go:1386: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1386
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1127
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:672
        	Error:      	Not equal: 
        	            	expected: "\n=== Without DATABRICKS_CONFIG_FILE the server is added for the default config file\n\n>>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code\nProfile logfood ([DATABRICKS_URL])\nAgents (named with --agents): claude-code\n\ngenie_one_mcp-logfood  system.ai.genie_one_mcp\n  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json\n\n=== Claude Code config has no pinned variables for the server\n\n>>> gron.py home/.claude.json\njson.mcpServers.genie_one_mcp-logfood.env = {};\n\n=== State holds the binding with no config file\n\n>>> jq .mcp.servers[] | {name, profile, config_file} home/.databricks/aitools/skills/.state.json\n{\n  \"name\": \"genie_one_mcp-logfood\",\n  \"profile\": \"logfood\",\n  \"config_file\": \"\"\n}\n"
        	            	actual  : "\n=== Without DATABRICKS_CONFIG_FILE the server is added for the default config file\n\n>>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code\nError: unknown flag: --names\n\nUsage:\n  databricks aitools [command]\n\nAvailable Commands:\n  install     Install Databricks skills and plugins for coding agents\n  list        List installed skills and plugins\n  uninstall   Uninstall Databricks skills and plugins\n  update      Update installed skills and plugins\n  version     Show installed skills and plugins version\n\nFlags:\n  -h, --help   help for aitools\n\nGlobal Flags:\n      --debug            enable debug logging\n  -o, --output type      output type: text or json (default text)\n  -p, --profile string   ~/.databrickscfg profile\n  -t, --target string    bundle target to use (if applicable)\n\nUse \"databricks aitools [command] --help\" for more information about a command.\n\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,21 +4,27 @@
        	            	 >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code
        	            	-Profile logfood ([DATABRICKS_URL])
        	            	-Agents (named with --agents): claude-code
        	            	+Error: unknown flag: --names
        	            	 
        	            	-genie_one_mcp-logfood  system.ai.genie_one_mcp
        	            	-  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json
        	            	+Usage:
        	            	+  databricks aitools [command]
        	            	 
        	            	-=== Claude Code config has no pinned variables for the server
        	            	+Available Commands:
        	            	+  install     Install Databricks skills and plugins for coding agents
        	            	+  list        List installed skills and plugins
        	            	+  uninstall   Uninstall Databricks skills and plugins
        	            	+  update      Update installed skills and plugins
        	            	+  version     Show installed skills and plugins version
        	            	 
        	            	->>> gron.py home/.claude.json
        	            	-json.mcpServers.genie_one_mcp-logfood.env = {};
        	            	+Flags:
        	            	+  -h, --help   help for aitools
        	            	 
        	            	-=== State holds the binding with no config file
        	            	+Global Flags:
        	            	+      --debug            enable debug logging
        	            	+  -o, --output type      output type: text or json (default text)
        	            	+  -p, --profile string   ~/.databrickscfg profile
        	            	+  -t, --target string    bundle target to use (if applicable)
        	            	 
        	            	->>> jq .mcp.servers[] | {name, profile, config_file} home/.databricks/aitools/skills/.state.json
        	            	-{
        	            	-  "name": "genie_one_mcp-logfood",
        	            	-  "profile": "logfood",
        	            	-  "config_file": ""
        	            	-}
        	            	+Use "databricks aitools [command] --help" for more information about a command.
        	            	 
        	            	+
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/mcp/add-default-config
        	Messages:   	cmd/aitools/mcp/add-default-config/output.txt vs /tmp/TestAcceptcmdaitoolsmcpadd-default-config1554218385/002/output.txt
    acceptance_test.go:1164: 
        LOG.create-genie_one_mcp-logfood: {
        LOG.create-genie_one_mcp-logfood:   "comment": "Ask questions about your data",
        LOG.create-genie_one_mcp-logfood:   "created_by": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "metastore_id": "7a0ca87c-3c44-ba65-7a0c-a87c3c44bb33",
        LOG.create-genie_one_mcp-logfood:   "name": "mcp-services/system.ai.genie_one_mcp",
        LOG.create-genie_one_mcp-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-genie_one_mcp-logfood: }
TestAccept/cmd/aitools/mcp/add-remove-agent-clis: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1386: Diff:
        --- cmd/aitools/mcp/add-remove-agent-clis/output.txt
        +++ /tmp/TestAcceptcmdaitoolsmcpadd-remove-agent-clis2219537333/002/output.txt
        @@ -2,388 +2,28 @@
         === add goes through each agent's own CLI: the stand-ins are the only agent CLIs on PATH
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,codex,gemini,copilot --output json
        -{
        -  "profile": "logfood",
        -  "host": "[DATABRICKS_URL]",
        -  "scope": "global",
        -  "servers": [
        -    {
        -      "name": "genie_one_mcp-logfood",
        -      "service": "system.ai.genie_one_mcp",
        -      "sign_in_required": false,
        -      "agents": [
        -        {
        -          "name": "claude-code",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.claude.json"
        -        },
        -        {
        -          "name": "codex",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.codex/config.toml"
        -        },
        -        {
        -          "name": "gemini",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.gemini/settings.json"
        -        },
        -        {
        -          "name": "copilot",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.copilot/mcp-config.json"
        -        }
        -      ]
        -    },
        -    {
        -      "name": "github-logfood",
        -      "service": "system.ai.github",
        -      "sign_in_required": true,
        -      "agents": [
        -        {
        -          "name": "claude-code",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.claude.json"
        -        },
        -        {
        -          "name": "codex",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.codex/config.toml"
        -        },
        -        {
        -          "name": "gemini",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.gemini/settings.json"
        -        },
        -        {
        -          "name": "copilot",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.copilot/mcp-config.json"
        -        }
        -      ]
        -    }
        -  ],
        -  "error_category": "TYPE_UNSPECIFIED"
        -}
        +Error: unknown flag: --names
         
        -=== the argv each agent CLI ran for add
        +Usage:
        +  databricks aitools [command]
         
        ->>> cat agent-calls.log
        -claude
        -	mcp
        -	add-json
        -	--scope
        -	user
        -	genie_one_mcp-logfood
        -	{"args":["aitools","mcp","serve","--profile","logfood","--service","system.ai.genie_one_mcp","--agent","claude-code"],"command":"[CLI]","env":{"DATABRICKS_CONFIG_FILE":"[TEST_TMP_DIR]/.databrickscfg"},"type":"stdio"}
        -codex
        -	mcp
        -	add
        -	genie_one_mcp-logfood
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.genie_one_mcp
        -	--agent
        -	codex
        -gemini
        -	mcp
        -	add
        -	--scope
        -	user
        -	-e
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	genie_one_mcp-logfood
        -	[CLI]
        -	--
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.genie_one_mcp
        -	--agent
        -	gemini
        -copilot
        -	mcp
        -	add
        -	genie_one_mcp-logfood
        -	--tools
        -	*
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.genie_one_mcp
        -	--agent
        -	copilot
        -claude
        -	mcp
        -	add-json
        -	--scope
        -	user
        -	github-logfood
        -	{"args":["aitools","mcp","serve","--profile","logfood","--service","system.ai.github","--agent","claude-code"],"command":"[CLI]","env":{"DATABRICKS_CONFIG_FILE":"[TEST_TMP_DIR]/.databrickscfg"},"type":"stdio"}
        -codex
        -	mcp
        -	add
        -	github-logfood
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.github
        -	--agent
        -	codex
        -gemini
        -	mcp
        -	add
        -	--scope
        -	user
        -	-e
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	github-logfood
        -	[CLI]
        -	--
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.github
        -	--agent
        -	gemini
        -copilot
        -	mcp
        -	add
        -	github-logfood
        -	--tools
        -	*
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.github
        -	--agent
        -	copilot
... (237 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.26.0):

acceptance_test.go:1386: Diff:
        --- cmd/aitools/mcp/add-remove-agent-clis/output.txt
        +++ /tmp/TestAcceptcmdaitoolsmcpadd-remove-agent-clis3556070248/002/output.txt
        @@ -2,388 +2,28 @@
         === add goes through each agent's own CLI: the stand-ins are the only agent CLIs on PATH
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,codex,gemini,copilot --output json
        -{
        -  "profile": "logfood",
        -  "host": "[DATABRICKS_URL]",
        -  "scope": "global",
        -  "servers": [
        -    {
        -      "name": "genie_one_mcp-logfood",
        -      "service": "system.ai.genie_one_mcp",
        -      "sign_in_required": false,
        -      "agents": [
        -        {
        -          "name": "claude-code",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.claude.json"
        -        },
        -        {
        -          "name": "codex",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.codex/config.toml"
        -        },
        -        {
        -          "name": "gemini",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.gemini/settings.json"
        -        },
        -        {
        -          "name": "copilot",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.copilot/mcp-config.json"
        -        }
        -      ]
        -    },
        -    {
        -      "name": "github-logfood",
        -      "service": "system.ai.github",
        -      "sign_in_required": true,
        -      "agents": [
        -        {
        -          "name": "claude-code",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.claude.json"
        -        },
        -        {
        -          "name": "codex",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.codex/config.toml"
        -        },
        -        {
        -          "name": "gemini",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.gemini/settings.json"
        -        },
        -        {
        -          "name": "copilot",
        -          "status": "installed",
        -          "path": "[TEST_TMP_DIR]/home/.copilot/mcp-config.json"
        -        }
        -      ]
        -    }
        -  ],
        -  "error_category": "TYPE_UNSPECIFIED"
        -}
        +Error: unknown flag: --names
         
        -=== the argv each agent CLI ran for add
        +Usage:
        +  databricks aitools [command]
         
        ->>> cat agent-calls.log
        -claude
        -	mcp
        -	add-json
        -	--scope
        -	user
        -	genie_one_mcp-logfood
        -	{"args":["aitools","mcp","serve","--profile","logfood","--service","system.ai.genie_one_mcp","--agent","claude-code"],"command":"[CLI]","env":{"DATABRICKS_CONFIG_FILE":"[TEST_TMP_DIR]/.databrickscfg"},"type":"stdio"}
        -codex
        -	mcp
        -	add
        -	genie_one_mcp-logfood
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.genie_one_mcp
        -	--agent
        -	codex
        -gemini
        -	mcp
        -	add
        -	--scope
        -	user
        -	-e
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	genie_one_mcp-logfood
        -	[CLI]
        -	--
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.genie_one_mcp
        -	--agent
        -	gemini
        -copilot
        -	mcp
        -	add
        -	genie_one_mcp-logfood
        -	--tools
        -	*
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.genie_one_mcp
        -	--agent
        -	copilot
        -claude
        -	mcp
        -	add-json
        -	--scope
        -	user
        -	github-logfood
        -	{"args":["aitools","mcp","serve","--profile","logfood","--service","system.ai.github","--agent","claude-code"],"command":"[CLI]","env":{"DATABRICKS_CONFIG_FILE":"[TEST_TMP_DIR]/.databrickscfg"},"type":"stdio"}
        -codex
        -	mcp
        -	add
        -	github-logfood
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.github
        -	--agent
        -	codex
        -gemini
        -	mcp
        -	add
        -	--scope
        -	user
        -	-e
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	github-logfood
        -	[CLI]
        -	--
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.github
        -	--agent
        -	gemini
        -copilot
        -	mcp
        -	add
        -	github-logfood
        -	--tools
        -	*
        -	--env
        -	DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg
        -	--
        -	[CLI]
        -	aitools
        -	mcp
        -	serve
        -	--profile
        -	logfood
        -	--service
        -	system.ai.github
        -	--agent
        -	copilot
... (237 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)
TestAccept/cmd/aitools/mcp/add-text: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1386: Diff:
        --- cmd/aitools/mcp/add-text/output.txt
        +++ /tmp/TestAcceptcmdaitoolsmcpadd-text2742010575/002/output.txt
        @@ -2,73 +2,28 @@
         === Text output: Cursor approval hint, and the sign-in step for github
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,cursor
        -Profile logfood ([DATABRICKS_URL])
        -Agents (named with --agents): claude-code, cursor
        +Error: unknown flag: --names
         
        -genie_one_mcp-logfood  system.ai.genie_one_mcp
        -  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       installed  [TEST_TMP_DIR]/home/.cursor/mcp.json
        -Cursor may ask you to approve genie_one_mcp-logfood. Run: cursor-agent mcp enable genie_one_mcp-logfood (the Cursor docs call the CLI agent), or approve it in Cursor's MCP settings.
        +Usage:
        +  databricks aitools [command]
         
        -github-logfood  system.ai.github
        -  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       installed  [TEST_TMP_DIR]/home/.cursor/mcp.json
        -system.ai.github needs sign-in. Run: DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg databricks aitools mcp login system.ai.github --profile logfood
        -Cursor may ask you to approve github-logfood. Run: cursor-agent mcp enable github-logfood (the Cursor docs call the CLI agent), or approve it in Cursor's MCP settings.
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
         
        -=== Running it again re-asserts the entries, and nothing is written twice
        +Flags:
        +  -h, --help   help for aitools
         
        ->>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,cursor
        -Profile logfood ([DATABRICKS_URL])
        -Agents (named with --agents): claude-code, cursor
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        -genie_one_mcp-logfood  system.ai.genie_one_mcp
        -  claude-code  already installed, re-asserted  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       already installed, re-asserted  [TEST_TMP_DIR]/home/.cursor/mcp.json
        +Use "databricks aitools [command] --help" for more information about a command.
         
        -github-logfood  system.ai.github
        -  claude-code  already installed, re-asserted  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       already installed, re-asserted  [TEST_TMP_DIR]/home/.cursor/mcp.json
        -system.ai.github needs sign-in. Run: DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg databricks aitools mcp login system.ai.github --profile logfood
         
        -=== Cursor config
        -
        ->>> cat home/.cursor/mcp.json
        -{
        -  "mcpServers": {
        -    "genie_one_mcp-logfood": {
        -      "args": [
        -        "aitools",
        -        "mcp",
        -        "serve",
        -        "--profile",
        -        "logfood",
        -        "--service",
        -        "system.ai.genie_one_mcp",
        -        "--agent",
        -        "cursor"
        -      ],
        -      "command": "[CLI]",
        -      "env": {
        -        "DATABRICKS_CONFIG_FILE": "[TEST_TMP_DIR]/.databrickscfg"
        -      }
        -    },
        -    "github-logfood": {
        -      "args": [
        -        "aitools",
        -        "mcp",
        -        "serve",
        -        "--profile",
        -        "logfood",
        -        "--service",
        -        "system.ai.github",
        -        "--agent",
        -        "cursor"
        -      ],
        -      "command": "[CLI]",
        -      "env": {
        -        "DATABRICKS_CONFIG_FILE": "[TEST_TMP_DIR]/.databrickscfg"
        -      }
        -    }
        -  }
        -}
        +Exit code: 1
        
    acceptance_test.go:1164: 
        LOG.create-genie_one_mcp-logfood: {
        LOG.create-genie_one_mcp-logfood:   "comment": "Ask questions about your data",
        LOG.create-genie_one_mcp-logfood:   "created_by": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "metastore_id": "7a0ca874-7b68-20a1-7a0c-a8747b68227e",
        LOG.create-genie_one_mcp-logfood:   "name": "mcp-services/system.ai.genie_one_mcp",
        LOG.create-genie_one_mcp-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-genie_one_mcp-logfood: }
    acceptance_test.go:1164: 
        LOG.create-github-logfood: {
        LOG.create-github-logfood:   "comment": "Repositories, issues, pull requests",
        LOG.create-github-logfood:   "config": {
        LOG.create-github-logfood:     "source_connection": {
        LOG.create-github-logfood:       "name": "connections/github",
        LOG.create-github-logfood:       "options": {
        LOG.create-github-logfood:         "oauth_provider": "OAUTH_PROVIDER_GITHUB_MCP"
        LOG.create-github-logfood:       }
        LOG.create-github-logfood:     }
        LOG.create-github-logfood:   },
        LOG.create-github-logfood:   "created_by": "tester@databricks.com",
        LOG.create-github-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-github-logfood:   "metastore_id": "7a0ca874-7c9a-ea18-7a0c-a8747c9aeae7",
        LOG.create-github-logfood:   "name": "mcp-services/system.ai.github",
        LOG.create-github-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-github-logfood: }

latest (v1.26.0):

acceptance_test.go:1386: Diff:
        --- cmd/aitools/mcp/add-text/output.txt
        +++ /tmp/TestAcceptcmdaitoolsmcpadd-text1806263285/002/output.txt
        @@ -2,73 +2,28 @@
         === Text output: Cursor approval hint, and the sign-in step for github
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,cursor
        -Profile logfood ([DATABRICKS_URL])
        -Agents (named with --agents): claude-code, cursor
        +Error: unknown flag: --names
         
        -genie_one_mcp-logfood  system.ai.genie_one_mcp
        -  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       installed  [TEST_TMP_DIR]/home/.cursor/mcp.json
        -Cursor may ask you to approve genie_one_mcp-logfood. Run: cursor-agent mcp enable genie_one_mcp-logfood (the Cursor docs call the CLI agent), or approve it in Cursor's MCP settings.
        +Usage:
        +  databricks aitools [command]
         
        -github-logfood  system.ai.github
        -  claude-code  installed  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       installed  [TEST_TMP_DIR]/home/.cursor/mcp.json
        -system.ai.github needs sign-in. Run: DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg databricks aitools mcp login system.ai.github --profile logfood
        -Cursor may ask you to approve github-logfood. Run: cursor-agent mcp enable github-logfood (the Cursor docs call the CLI agent), or approve it in Cursor's MCP settings.
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
         
        -=== Running it again re-asserts the entries, and nothing is written twice
        +Flags:
        +  -h, --help   help for aitools
         
        ->>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,cursor
        -Profile logfood ([DATABRICKS_URL])
        -Agents (named with --agents): claude-code, cursor
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        -genie_one_mcp-logfood  system.ai.genie_one_mcp
        -  claude-code  already installed, re-asserted  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       already installed, re-asserted  [TEST_TMP_DIR]/home/.cursor/mcp.json
        +Use "databricks aitools [command] --help" for more information about a command.
         
        -github-logfood  system.ai.github
        -  claude-code  already installed, re-asserted  [TEST_TMP_DIR]/home/.claude.json
        -  cursor       already installed, re-asserted  [TEST_TMP_DIR]/home/.cursor/mcp.json
        -system.ai.github needs sign-in. Run: DATABRICKS_CONFIG_FILE=[TEST_TMP_DIR]/.databrickscfg databricks aitools mcp login system.ai.github --profile logfood
         
        -=== Cursor config
        -
        ->>> cat home/.cursor/mcp.json
        -{
        -  "mcpServers": {
        -    "genie_one_mcp-logfood": {
        -      "args": [
        -        "aitools",
        -        "mcp",
        -        "serve",
        -        "--profile",
        -        "logfood",
        -        "--service",
        -        "system.ai.genie_one_mcp",
        -        "--agent",
        -        "cursor"
        -      ],
        -      "command": "[CLI]",
        -      "env": {
        -        "DATABRICKS_CONFIG_FILE": "[TEST_TMP_DIR]/.databrickscfg"
        -      }
        -    },
        -    "github-logfood": {
        -      "args": [
        -        "aitools",
        -        "mcp",
        -        "serve",
        -        "--profile",
        -        "logfood",
        -        "--service",
        -        "system.ai.github",
        -        "--agent",
        -        "cursor"
        -      ],
        -      "command": "[CLI]",
        -      "env": {
        -        "DATABRICKS_CONFIG_FILE": "[TEST_TMP_DIR]/.databrickscfg"
        -      }
        -    }
        -  }
        -}
        +Exit code: 1
        
    acceptance_test.go:1164: 
        LOG.create-genie_one_mcp-logfood: {
        LOG.create-genie_one_mcp-logfood:   "comment": "Ask questions about your data",
        LOG.create-genie_one_mcp-logfood:   "created_by": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-genie_one_mcp-logfood:   "metastore_id": "7a0ca87c-337b-ec77-7a0c-a87c337bed2a",
        LOG.create-genie_one_mcp-logfood:   "name": "mcp-services/system.ai.genie_one_mcp",
        LOG.create-genie_one_mcp-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-genie_one_mcp-logfood: }
    acceptance_test.go:1164: 
        LOG.create-github-logfood: {
        LOG.create-github-logfood:   "comment": "Repositories, issues, pull requests",
        LOG.create-github-logfood:   "config": {
        LOG.create-github-logfood:     "source_connection": {
        LOG.create-github-logfood:       "name": "connections/github",
        LOG.create-github-logfood:       "options": {
        LOG.create-github-logfood:         "oauth_provider": "OAUTH_PROVIDER_GITHUB_MCP"
        LOG.create-github-logfood:       }
        LOG.create-github-logfood:     }
        LOG.create-github-logfood:   },
        LOG.create-github-logfood:   "created_by": "tester@databricks.com",
        LOG.create-github-logfood:   "effective_owner": "tester@databricks.com",
        LOG.create-github-logfood:   "metastore_id": "7a0ca87c-355d-e1e7-7a0c-a87c355de2b1",
        LOG.create-github-logfood:   "name": "mcp-services/system.ai.github",
        LOG.create-github-logfood:   "updated_by": "tester@databricks.com"
        LOG.create-github-logfood: }

16 more failure details omitted, see the full report in the run.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: f20df6b

Run: 43467516090

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 35 5:40
💚​ aws-windows-latest-4core-16gb 1 272 33 5:31
💚​ azure linux-2core-8gb 1 269 35 5:30
💚​ azure-windows-latest-4core-16gb 1 271 33 5:33
💚​ gcp linux-2core-8gb 1 270 35 6:00
💚​ gcp-windows-latest-4core-16gb 1 272 33 5:33
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:21 gcp-windows-latest-4core-16gb TestAccept
3:56 azure-windows-latest-4core-16gb TestAccept
3:43 aws-windows-latest-4core-16gb TestAccept

@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 2dd54b7 to a1fd867 Compare November 12, 2026 14:44
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 62533cc to a1d0d64 Compare November 12, 2026 14:47
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from a1fd867 to b90afff Compare November 12, 2026 17:15
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from a1d0d64 to 55e2a1d Compare November 12, 2026 17:16
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from b90afff to 0aa2ee4 Compare November 13, 2026 10:45
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 55e2a1d to a603d10 Compare November 13, 2026 11:06
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 0aa2ee4 to 2dbdfe3 Compare November 13, 2026 12:28
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from a603d10 to 1391f64 Compare November 13, 2026 12:28
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 1fff1da to e9bad5b Compare November 13, 2026 13:28
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 1391f64 to c3e93cc Compare November 13, 2026 14:27
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from b30caa0 to 43a64fd Compare November 13, 2026 15:06
simonfaltum and others added 10 commits November 13, 2026 15:09
@simonfaltum Isaac
Log AitoolsMcpEvent from aitools mcp and the serve proxy
ca772f4 
The MCP commands and the proxy now buffer an AitoolsMcpEvent for the
telemetry upload at exit: one result per (service, agent) for add and
remove, one per service for login and logout, and PROXY_STARTED and
SIGN_IN_PROMPTED from serve. Events carry enums and counts only; no host,
profile, workspace ID, custom service or schema name, path or error text.

The event mirrors the lumberjack schema for aitools MCP, which is not merged
yet. The enum values for the MCP error categories are added next to the
install ones, and the commands use them in place of local constants.
Profile, sign-in and list failures that had no typed category now report
NO_PROFILE, PROFILE_REQUIRED, MCP_PERMISSION_DENIED, MCP_LOCATION_NOT_FOUND
and MCP_NAME_CONFLICT.

The fake /telemetry-ext now reports the batch size, as the server does.
Before, a batch of two or more events was resent by the CLI's retry.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Create the agent config dirs in the telemetry acceptance cases
635ce6d 
mcp add now refuses an agent named with --agents whose config directory
is missing. These cases add for claude-code (and codex in mcp-add) in a
fresh HOME, so they create the directories first. The telemetry events
do not change.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Record the sign-in prompt only when a page URL was shown
b6f82a6 
A sign-in that fails before its URL is printed, such as a read of the
workspace ID, prompted nobody, so it no longer logs SIGN_IN_PROMPTED.
The serve command sets the proxy's recorder where it builds the
challenge, so the challenge is never handed over without one.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Keep one set of login steps on an MCP authentication failure
231e0b4 
The MCP sign-in commands set their config for the telemetry upload, which
also turns on root's auth enrichment. An AUTH_FAILED error already names
the login command for its profile, so enrichment now leaves such an error
as it is, and the user gets one set of next steps.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Resolve the remove upload's auth after the local edit
a87a80d 
remove resolved the profile before it touched any file, so an offline run
could wait on the network first, even with telemetry disabled. The edit now
runs first; the upload's profile is resolved afterwards, with a three second
budget, and only when telemetry is on.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Log serve's start after its checks, and keep the failed start's upload
3452750 
PROXY_STARTED was logged before the startup checks, so a start the lookup
rejected uploaded as a clean start. It is now logged once the checks have
run, as AUTH_FAILED when they fail. A binding that fails AUTH_FAILED
uploads with its profile, whose host the binding has already checked. A
profile whose host changed still uploads nothing, since the binding never
contacts a host the record does not name. A rejected refresh token clears
the upload config, so the exit upload does not ask the token endpoint again.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Log only events with a reached server, and count profiles by config file
2bd5d17 
Login and logout log nothing when no server was reached, since their upload
has no config. A sign-in state that was never learned is left unset rather
than PENDING. A remove that only drops records logs its REMOVE, since the
state changes. add --login logs SIGN_IN UNCHANGED when the credential was
already active and the question was skipped. The bound profile count keys
profiles by config file as well as name, as list does.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Derive the telemetry service enum from the known system.ai table
0f3e651 
mcpServiceType parsed the system.ai schema name itself and repeated the
table's leaves. It now uses the shared schema constant and the known table,
so an unknown system.ai leaf is OTHER_SYSTEM and a service outside system.ai
is CUSTOM. A test checks that every known leaf has its own enum.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Tidy the telemetry acceptance scripts and the fake telemetry endpoint
61fdc2c 
The four telemetry scripts share one mcp_events helper that filters the
uploads with print_requests.py. Their API posts carry MSYS_NO_PATHCONV so
Windows keeps the path, the required failures use musterr, the invalid
Codex config is a committed input, and the CLI calls are traced. The fake
/telemetry-ext reports an unreadable batch as an error, as its neighbors do.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Document the error categories of the server result and tidy its test
48880f0 
The server result's doc now names the categories a proxy start and a
SIGN_IN_PROMPTED carry. The user-content test uses slices and maps from the
standard library, and its comment no longer claims a host check the test
does not make.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from f6918d2 to 48880f0 Compare November 13, 2026 15:12
simonfaltum and others added 6 commits November 13, 2026 15:33
@simonfaltum Isaac
Authenticate telemetry uploads without refreshing a token
b50fd13 
The config of a telemetry upload that the command does not otherwise use
never refreshes its token: the command may exit before a refresh finishes,
and a rotated single-use refresh token would be lost. serve's session keeps
its refreshing config, and its upload gets a config of its own.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Resolve remove's telemetry auth only after it logs an event
b5918d0 
remove writes its output first, then resolves the auth of its upload, only
when a server was touched. The upload is skipped when a server recorded for
its profile no longer matches the profile's host, workspace or config file,
so the token is not sent to a host the server was not set up against. The
auth resolution is the one install uses, logged at debug level and bounded
by the same budget.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Resolve add, login and logout telemetry auth as install does
8c93399 
The three commands no longer set their profile's config for the upload
themselves. withTelemetryAuth resolves it in the background, as install
does, and the run waits for it once it has logged an event. The events
upload with the auth root resolves, so root's auth enrichment still applies
to their errors; the MCP login hint stays the only set of next steps.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Log PROXY_STARTED only for a serving start
76a784d 
A start that fails its checks has no upload to make: a binding that failed
has no profile the upload may authenticate with, and a rejected credential
would only repeat its rejected request. serve logs its start once the
checks have passed, and an event that names a rejected credential is not
logged.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Tidy the MCP telemetry: sign-in and record-only states, service enum,…
e98a36a 
… docs

An add whose sign-in failed before reading a state reports PENDING. A
record-only removal that fails to save is reported FAILED. The service enum
is one lookup that falls back to OTHER_SYSTEM. The server result doc says
which outcomes carry an error category. Test comments name their test.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Upload telemetry with a config the command sets
2ab75de 
The upload authenticates with a config a command sets once its binding checks have passed, in place of the one
cmdctx.ConfigUsed holds, and a run can refuse its upload after a rejected credential. Commands that bind a profile
use this instead of resolving the profile's auth in the background.

Co-authored-by: Isaac <no-reply@databricks.com>
simonfaltum and others added 6 commits November 13, 2026 16:19
@simonfaltum Isaac
Authenticate MCP telemetry uploads with the profile each command binds
8a5baa2 
add, login, logout, remove and serve no longer resolve the upload's auth in the background, as install does. Each one
sets the config of the profile it bound once its binding checks have passed, and an upload never refreshes a token.
remove sets it only when the removed servers share one profile and none of the servers recorded for it has drifted.
serve sets it at exit, after its refresh guard has run, and a rejected credential refuses the upload.

Errors of these commands are no longer enriched by the root command, since their upload no longer sets the command's
config. The auth error hint that skipped that enrichment, and cmdctx.ClearConfigUsed, are removed. A removal that
prunes an entry is reported CHANGED, and a failed state save reports it FAILED.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Record sign-in prompts of waits on a cancelled page, and test the ref…
a40ef5c 
…usal and removal outcomes

An authenticate call that waits on the page an earlier cancelled call showed records its outcome as a prompt too, so a
sign-in that completes there is ACTIVE rather than PENDING. The proto mirror says a retry reports its own outcome.

Tests cover the upload refusal after a rejected credential, a pruned entry reported CHANGED (FAILED when the state is
not saved), and a failed state read after another event.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Tidy the MCP telemetry: one SIGN_IN builder, rejected-start test, doc…
be0e3ba 
…umented enum mirrors

signInEventOf is the one SIGN_IN event builder, used by add --login and login. A misplaced test doc comment moves to its
test, and remove's doc says the edit needs no network. The rejected-start case of the telemetry acceptance tests is
replaced by a unit test that a rejected startup read logs nothing. KnownLeaves has its own test, and the enum values
that no code names have comments that they mirror the schema.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Refuse the MCP telemetry upload after the service rejects the credential
1174ea7 
A 401 that does not ask for a sign-in is answered with the auth failure, but the run's exit-time upload still
authenticated with the credential the workspace had just rejected. The proxy now reports that answer to the
command, which refuses the upload as a refresh failure does.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Keep the telemetry upload's host metadata failures out of the command…
a1d30e0 
… output

The SDK logs a failed host metadata lookup as a warning through its global logger, and the upload's resolution
runs on a background context that reaches it. A command that succeeded then printed a warning that only its
telemetry upload caused. The upload's resolver now logs the failure at debug and falls back to the user config,
as the SDK does.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum Isaac
Group the MCP telemetry profile count as mcp list does, and tidy the …
f20df6b 
…MCP helpers

boundProfileCount now groups servers through the same helper mcp list uses, so the count matches the listing in
every case, including a config file that does not resolve. KnownLeaves is marked as kept for its tests, the
sign-in constructors return their command directly, the pruned-install test setup is a named helper, and the
enum comments say that the schema mirrors are not sent by the CLI.

Co-authored-by: Isaac <no-reply@databricks.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lennartkats-db lennartkats-db Awaiting requested review from lennartkats-db lennartkats-db will be requested when the pull request is marked ready for review lennartkats-db is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@simonfaltum @eng-dev-ecosystem-bot