Skip to content

Add MCP state records, drift check and profile binding checks - #7537

Draft
simonfaltum wants to merge 7 commits into
aitools-mcp/02-discoveryfrom
aitools-mcp/03-state
Draft

simonfaltum wants to merge 7 commits into
aitools-mcp/02-discoveryfrom
aitools-mcp/03-state

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

This pull request is part of a stack, on top of #7538, that adds the hidden databricks aitools mcp command group. It adds MCP state records, drift checks and profile binding checks. Nothing calls them yet.

Changes

  • MCP records are the mcp section of the aitools state file (schema v3). A file with only MCP records counts as no install. Installer saves keep the section. Uninstall and SaveMCPState remove the file once nothing remains.
  • Reconcile reports profile_missing and host_changed, using the host and workspace the SDK resolves, with no network.
  • Bind checks the profile before it resolves credentials, so a re-pointed host is never contacted.
  • SDK configuration errors are UNCATEGORIZED. Only failed authentication is AUTH_FAILED. BindError.ExitCode() returns 2, which the serve PR maps to the process exit status.

Why

A proxy must refuse to start when its profile was deleted or re-pointed, or an agent would silently use the wrong workspace. It must never send another workspace's credential.

Decisions to review

  • MCP records share the skills state file. A CLI older than schema v3 drops the mcp section on its next save, and its uninstall deletes a file that holds only MCP records.
  • An entry pinned to another config file fails the bind with PROFILE_HOST_CHANGED, rather than printing the no-record notice, because binding through this file would switch the server silently. Changing DATABRICKS_CONFIG_FILE means removing and adding the server again.
  • An entry with no pinned config file is checked against ~/.databrickscfg, not DATABRICKS_CONFIG_FILE.
  • Binding never reads DATABRICKS_TOKEN or other auth variables into the profile. A profile without a token uses the SDK chain for its host.
  • Host comparison ignores case, unlike profile.WithHost. A profile that drops its workspace selector is not detected, since the recorded ID may come from the API.

Tests

  • TestBindHostChangedDoesNotContactNewHost fails if a re-pointed host gets a request. TestBindWorkspaceFollowsSDKResolution covers the host query.
  • TestBindEntryInOtherConfigFileFailsClosed and TestBindNeverUsesEnvTokenForProfileWithoutToken cover the config-file and credential rules.
  • TestUninstallRemovesLeftoverFilesWithoutSkills, TestUpdateAfterPruningEverySkillAddsNewSkills and TestInstallSpecificSkillWithEmptyStateFile pass on main's behavior, which users without MCP records keep.
  • TestLoadStateHidesOnlyMCPOnlyFiles and TestUninstallKeepsStateFileWithMCPRecords cover the shared file.
  • Ran: race tests on the touched packages, experimental/aitools and cmd/aitools acceptance tests (goldens unchanged), GOOS=windows go vet, golangci-lint and task checks.
  • Not run: the Windows and macOS test suites. No acceptance test covers MCP-only output, since no CLI path writes MCP records yet.

This pull request and its description were written by Isaac.

@simonfaltum
simonfaltum changed the base branch from main to aitools-mcp/02-discovery November 12, 2026 11:49
@simonfaltum
simonfaltum force-pushed the aitools-mcp/03-state branch 2 times, most recently from e003f2d to 99f703a Compare November 12, 2026 11:57
@simonfaltum
simonfaltum force-pushed the aitools-mcp/02-discovery branch from 773f68b to 93a7090 Compare November 12, 2026 14:24
@simonfaltum simonfaltum changed the title Add MCP state file, drift check, and profile binding checks Add MCP state records, drift check and profile binding checks Nov 12, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 03c01a0

Run: 43449399633

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 14:00
💚​ aws-windows-latest-4core-16gb 1 272 13 7:37
💚​ azure linux-2core-8gb 1 269 15 10:44
💚​ azure-windows-latest-4core-16gb 1 271 13 7:39
💚​ gcp linux-2core-8gb 1 270 15 11:24
💚​ gcp-windows-latest-4core-16gb 1 272 13 8:55
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:25 gcp-windows-latest-4core-16gb TestAccept
3:18 aws-windows-latest-4core-16gb TestAccept
3:14 azure-windows-latest-4core-16gb TestAccept
2:13 aws linux-2core-8gb TestImportDirWithOverwriteFlag
2:12 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:08 aws linux-2core-8gb TestFilerReadWrite/workspace_files_extensions

simonfaltum and others added 7 commits November 13, 2026 14:41
Adds the state file that records MCP servers the CLI wrote, a drift check
against the Databricks profiles file, and the binding checks a stdio proxy
runs before it reads stdin. These are the building blocks for `aitools mcp`.

Co-authored-by: Isaac <no-reply@databricks.com>
Keep MCP records in the shared aitools state file as an optional mcp
section (schema v3). Uninstall keeps the file while MCP records remain.

Binding checks now compare the host and the workspace ID with the
profile before any credential is resolved, read the config file named
in the context, and never fill credentials from the environment. Drift
reconciliation checks each entry against its pinned config file, and
entries match on service, profile and config file.

Host comparison is case-insensitive. Error and notice messages no
longer end with a period, and paths are slash-separated.

Co-authored-by: Isaac <no-reply@databricks.com>
- Compare the workspace a profile names in its host query (?o=, ?w=,
  ?workspace_id=) as well as in workspace_id. "none" names no workspace.
- Compare config files by resolved absolute path. An entry with no pinned
  file is the default ~/.databrickscfg, and Reconcile checks it there.
- Treat a state file that holds only MCP records as having no skills or
  plugins in version, update, uninstall, scope detection and the legacy
  install guard.
- Installer saves keep the MCP section on disk. SaveMCPState replaces it.
  Uninstall re-reads the MCP section before it deletes the state file.
- Move the DATABRICKS_CONFIG_FILE name into mcpconfig.ConfigFileEnv, shared
  with the agent entry writer.
- Read the home directory only when a config file path needs it.

Co-authored-by: Isaac <no-reply@databricks.com>
- Derive the profile's host and workspace with the SDK's own resolution,
  without network: workspace_id first, then the ?w= query, then a numeric
  ?o= or ?workspace_id=. "none" names no workspace.
- Resolve the config file once. Symlinks are resolved before the path is
  made absolute, so ".." after a symlink matches the file the OS reads.
  Paths compare case-insensitively on Windows and macOS. The profile
  lookup, the entry match and the SDK all use that path.
- An entry whose "~" path cannot be resolved is skipped while another entry
  matches. It fails the bind only when nothing else matches. Reconcile
  reports the error for that entry and still checks the others.
- Share the per-entry drift check between Bind and Reconcile, and the
  profile lookup too. A section with no host is reported as having no host,
  not as missing.
- SDK configuration errors are UNCATEGORIZED with no login hint. Only
  failures from Authenticate are AUTH_FAILED.
- Export databrickscfg.ResolveConfigFilePath and reuse it.
- SaveState removes the state file when no skills, plugins, files, repo
  directories or MCP servers remain. Uninstall uses this instead of its own
  re-read of the MCP section.
- aitools list treats a state file with only MCP records as no install.
- Fix the spacing in the profile-gone message. Document ConfigFileEnv and
  the helpers that lacked comments.

Co-authored-by: Isaac <no-reply@databricks.com>
- Bind fails closed when an entry for the service and profile is pinned to
  another config file, instead of binding through the file in use.
- A profile with an empty host is reported as having no host.
- The recorded workspace is read from the recorded host's query, as the SDK
  resolves the profile's.
- Each config file is read once per check; Reconcile no longer takes a Profiler.
- A path through a missing directory is kept as written, and existing files are
  compared by identity.
- The recreate, remove and login hints quote their values, and pinned entries
  name their config file.
- A state file that holds only MCP records loads as no install, so installer
  and cmd behavior for users without MCP records is main's again. SaveState no
  longer removes the file; SaveMCPState and uninstall do.

Co-authored-by: Isaac <no-reply@databricks.com>
The two config-file mismatch tests built the expected message from the
native resolved path, while the message prints it with forward slashes,
so they failed on Windows.

Co-authored-by: Isaac <no-reply@databricks.com>
Main renamed UninstallSkillsOpts to UninstallSkillsWithResult, which returns
the result as well, and replaced the schema v2 constant with the current
schema version. Return the new signature, use the current schema version in
the test fixtures, and restore the state path that readState needs.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum
simonfaltum force-pushed the aitools-mcp/02-discovery branch from 93a7090 to a4521da Compare November 13, 2026 15:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants