Skip to content

Add MCP service discovery, known services and naming for aitools - #7538

Draft
simonfaltum wants to merge 4 commits into
aitools-mcp/01-agentconfigfrom
aitools-mcp/02-discovery
Draft

simonfaltum wants to merge 4 commits into
aitools-mcp/01-agentconfigfrom
aitools-mcp/02-discovery

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

This is part of a stack that adds the hidden databricks aitools mcp command group, on top of #7539. This PR adds the helpers that group needs to discover Databricks managed MCP services, name their servers and build their URLs. No command calls them yet.

Changes

  • Discover lists the services in a schema with ListMcpServicesAll (view=FULL, so the response carries each service's source connection). Services whose source connection is deleted are dropped.
  • A service needs sign-in when its source connection has an oauth_provider option.
  • The schema and leaf come from the name the API returns, which Unity Catalog lowercases, so Main.Tools or System.AI as a location work.
  • system.ai services get a label, summary and note from a static table. Other services carry only the API comment.
  • ServerName(fullName, profile) returns <leaf>-<profile> for system.ai and <catalog>-<schema>-<leaf>-<profile> elsewhere, leaving out parts that sanitize to nothing. Sanitize lowercases, maps other characters to -, collapses _ runs and trims - and _, so a name never contains __, which agents use to split mcp__<server>__<tool>. UniqueName returns the first free -2, -3, ... name for a caller-supplied taken check.
  • ServiceURL (path-escaped) and SignInURL build the MCP endpoint and the sign-in page URL.
  • Listing failures return *ListError with kinds MCP_LOCATION_NOT_FOUND, MCP_PERMISSION_DENIED and RATE_LIMITED. The message names the likely cause and keeps the server's message, because the SDK maps every 404, and a private-link 403, to the same sentinels. The 403 hint names the grants listing needs: USE_CATALOG on the catalog and USE_SCHEMA on the schema. A 404 ENDPOINT_NOT_FOUND, FEATURE_DISABLED or a private-link failure keeps only the server's message, because it is not about the schema or grants.
  • The testserver serves GET /api/2.1/unity-catalog/mcp-services, filtering by parent (in any casing) and omitting source connections unless view is FULL.

Why

Adding, listing and removing servers all need the same service list, names and sign-in check. With view=FULL the list response carries oauth_provider for external services, so sign-in detection needs no per-service request. Built-in service docs: https://docs.databricks.com/aws/en/agents/mcp-tools/built-in-mcp-services

Decisions to review

  • Discovery does not retry on HTTP 429. The SDK already retries 429 for up to five minutes, so another retry would double the wait.
  • The known table holds labels and notes only. Sign-in comes from the API, so a table entry cannot hide a service.
  • Known is set only for system.ai, so a customer schema's github service gets no table entry.
  • UniqueName takes a taken function instead of reading state, so this package does not depend on the state file.

Tests

  • Discovery tests run against an httptest server. They cover one call for one page, pagination, sign-in detection, deleted connections, system.ai-only table lookup, mixed-case locations, and 404 and 403 mapping with their messages. 429, the other sentinels and the workspace and network error codes are covered through mapListError.
  • TestKnownSystemAITable checks the 11 system.ai labels. TestSanitize, TestServerName and TestUniqueName cover naming, including _ runs. TestServiceURLEscapesTheName covers a # in a name.
  • Testserver tests cover FULL and BASIC list output and parent filtering.
  • go test -race on both packages, golangci-lint and GOOS=windows go vet pass.
  • A read-only listing of schemas/System.AI on a real workspace returned the same 11 services, with lowercase names, as schemas/system.ai.

This pull request and its description were written by Isaac.

@simonfaltum
simonfaltum changed the base branch from main to aitools-mcp/01-agentconfig November 12, 2026 11:48
@simonfaltum
simonfaltum force-pushed the aitools-mcp/02-discovery branch from 0510ae3 to 27c9441 Compare November 12, 2026 11:49
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: a4521da

Run: 43449188637

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 10:17
💚​ aws-windows-latest-4core-16gb 1 272 13 7:12
💚​ azure linux-2core-8gb 1 269 15 8:40
💚​ azure-windows-latest-4core-16gb 1 271 13 6:26
💚​ gcp linux-2core-8gb 1 270 15 11:44
💚​ gcp-windows-latest-4core-16gb 1 272 13 8:23
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:27 gcp-windows-latest-4core-16gb TestAccept
3:20 aws-windows-latest-4core-16gb TestAccept
3:18 azure-windows-latest-4core-16gb TestAccept
2:34 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:33 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:09 azure-windows-latest-4core-16gb TestExportDir

simonfaltum and others added 4 commits November 13, 2026 14:41
Add the libs/aitools/mcp helpers that the `databricks aitools mcp` commands will
use: Discover lists the MCP services in a schema with one paginated call and
reports sign-in requirements from the source connection options, a static table
describes the system.ai services, and naming and URL helpers build server names
and service and sign-in URLs.

The testserver gains GET /api/2.1/unity-catalog/mcp-services, which filters by
parent and drops source connections unless view is FULL.

Co-authored-by: Isaac <no-reply@databricks.com>
Take the schema and leaf from the name the API returns, so a mixed-case
--location still finds the leaf and the system.ai table. Name the grants
listing needs and keep the server's message in listing errors. Collapse
underscore runs in server names so agents cannot split them at "__".
Escape the service name in the endpoint path. Use ListMcpServicesAll and
drop table fields nothing reads.

Co-authored-by: Isaac <no-reply@databricks.com>
ServerName now takes the service full name, so the system.ai check comes
from one source in any casing, and parts that sanitize to nothing are
left out instead of leaving a stray "-". Listing errors for a missing
endpoint, a disabled feature or private link keep the server's message
without a schema or grants hint. The fake list handler matches the
parent in any casing and uses the SDK's view constant; its seed helper
moves into the test file.

Co-authored-by: Isaac <no-reply@databricks.com>
One return path for unclassified listing errors, doc comments on Unwrap and
the testserver helper, the docs URL for the system.ai table, and terminal-
friendly service notes.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum
simonfaltum force-pushed the aitools-mcp/01-agentconfig branch from 6f07e48 to 07cc225 Compare November 13, 2026 15:05
@simonfaltum
simonfaltum force-pushed the aitools-mcp/02-discovery branch from 93a7090 to a4521da Compare November 13, 2026 15:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants