Repository navigation
aitools: bound best-effort telemetry auth #7468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * Fix `databricks aitools install` waiting a long time on an unreachable or slow workspace while it resolves authentication for telemetry; that wait is now limited to a few seconds. ([#7468](https://github.com/databricks/cli/pull/7468)) |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
|
|
||
| === workspace unreachable: the background auth resolution prints nothing | ||
|
|
||
| >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code | ||
| Installing Databricks skills for Claude Code... | ||
| Using skills version test-ref | ||
| Fetching skills manifest... | ||
| Installed 1 skill. | ||
|
|
||
| === --debug still shows the hostmetadata line, at debug level |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Isolate HOME so the real machine's agents and skills are not seen. The | ||
| # committed home/.claude is what agent detection finds. | ||
| sethome home | ||
|
|
||
| # The workspace host does not resolve, so the background telemetry auth | ||
| # resolution fails to fetch host metadata and logs hostmetadata's warning. | ||
| # The skills still come from the mock server (DATABRICKS_SKILLS_BASE_URL). | ||
| export DATABRICKS_HOST=https://unreachable.test | ||
|
|
||
| title "workspace unreachable: the background auth resolution prints nothing" | ||
| trace $CLI aitools install --skills-only --scope=global --agents=claude-code | ||
|
|
||
| title "--debug still shows the hostmetadata line, at debug level" | ||
| # A fresh cache, so the first run's negative-cache entry does not skip the fetch. | ||
| export DATABRICKS_CACHE_DIR="$PWD/debug-cache" | ||
| trace $CLI aitools install --skills-only --scope=global --agents=claude-code --debug &> LOG.debug | ||
| contains.py 'Debug: [hostmetadata] failed to fetch host metadata for https://unreachable.test' '!Warn: [hostmetadata]' < LOG.debug > /dev/null | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Mock server replaces raw.githubusercontent.com for the skills manifest and files. | ||
| Env.DATABRICKS_SKILLS_BASE_URL = "$DATABRICKS_HOST" | ||
| Env.DATABRICKS_SKILLS_REF = "test-ref" | ||
| # The case needs the telemetry auth resolution to run, whatever the shell sets. | ||
| Env.DATABRICKS_CLI_DISABLE_TELEMETRY = "" | ||
|
|
||
| Ignore = [ | ||
| "home", | ||
| "debug-cache", | ||
| ] | ||
|
|
||
| [[Server]] | ||
| Pattern = "GET /test-ref/manifest.json" | ||
| Response.Body = ''' | ||
| { | ||
| "version": "2", | ||
| "updated_at": "2026-01-01T00:00:00Z", | ||
| "skills": { | ||
| "test-stable": { | ||
| "version": "1.0.0", | ||
| "description": "Stable test skill", | ||
| "files": ["SKILL.md"], | ||
| "repo_dir": "skills" | ||
| } | ||
| } | ||
| } | ||
| ''' | ||
|
|
||
| [[Server]] | ||
| Pattern = "GET /test-ref/skills/test-stable/SKILL.md" | ||
| Response.Body = '''--- | ||
| name: test-stable | ||
| --- | ||
|
|
||
| # Test stable skill | ||
| ''' |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
|
|
||
| === install resolves auth in the background: its install event is uploaded | ||
|
|
||
| >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code | ||
| Installing Databricks skills for Claude Code... | ||
| Using skills version test-ref | ||
| Fetching skills manifest... | ||
| Installed 1 skill. | ||
|
|
||
| >>> print_telemetry | ||
| {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}} | ||
|
|
||
| === databricks-cli auth with a valid cached token: the event is uploaded | ||
|
|
||
| >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code --profile test-profile | ||
| Installing Databricks skills for Claude Code... | ||
| Using skills version test-ref | ||
| Fetching skills manifest... | ||
| Installed 1 skill. | ||
|
|
||
| >>> print_telemetry | ||
| {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}} | ||
|
|
||
| === databricks-cli auth with an expired cached token: no refresh, nothing is uploaded | ||
|
|
||
| >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code --profile test-profile | ||
| Installing Databricks skills for Claude Code... | ||
| Using skills version test-ref | ||
| Fetching skills manifest... | ||
| Installed 1 skill. | ||
|
|
||
| >>> print_requests.py //oidc/v1/token //telemetry-ext | ||
|
|
||
| >>> gron.py [TEST_TMP_DIR]/home/.databricks/token-cache.json | ||
| json.tokens.test-profile.access_token = "expired-access-token"; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Isolate HOME so the real machine's agents and skills are not seen. The | ||
| # committed home/.claude is what agent detection finds. | ||
| sethome home | ||
|
|
||
| title "install resolves auth in the background: its install event is uploaded" | ||
| trace $CLI aitools install --skills-only --scope=global --agents=claude-code | ||
| trace print_telemetry | ||
|
|
||
| # A databricks-cli profile whose token comes from the CLI's token cache. | ||
| cat > "$HOME/.databrickscfg" <<ENDCFG | ||
| [test-profile] | ||
| host = $DATABRICKS_HOST | ||
| auth_type = databricks-cli | ||
| ENDCFG | ||
| unset DATABRICKS_HOST DATABRICKS_TOKEN | ||
| write_token_cache() { | ||
| mkdir -p "$HOME/.databricks" | ||
| cat > "$HOME/.databricks/token-cache.json" <<ENDCACHE | ||
| { | ||
| "version": 1, | ||
| "tokens": { | ||
| "test-profile": { | ||
| "access_token": "$1", | ||
| "token_type": "Bearer", | ||
| "refresh_token": "test-refresh-token", | ||
| "expiry": "$2" | ||
| } | ||
| } | ||
| } | ||
| ENDCACHE | ||
| } | ||
|
|
||
| title "databricks-cli auth with a valid cached token: the event is uploaded" | ||
| write_token_cache valid-access-token 2099-01-01T00:00:00Z | ||
| trace $CLI aitools install --skills-only --scope=global --agents=claude-code --profile test-profile | ||
| trace print_telemetry | ||
|
|
||
| title "databricks-cli auth with an expired cached token: no refresh, nothing is uploaded" | ||
| write_token_cache expired-access-token 2000-01-01T00:00:00Z | ||
| trace $CLI aitools install --skills-only --scope=global --agents=claude-code --profile test-profile | ||
| trace print_requests.py //oidc/v1/token //telemetry-ext | ||
| trace gron.py "$HOME/.databricks/token-cache.json" | grep access_token |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # Mock server replaces raw.githubusercontent.com for the skills manifest and files. | ||
| Env.DATABRICKS_SKILLS_BASE_URL = "$DATABRICKS_HOST" | ||
| Env.DATABRICKS_SKILLS_REF = "test-ref" | ||
|
|
||
| [[Server]] | ||
| Pattern = "GET /test-ref/manifest.json" | ||
| Response.Body = ''' | ||
| { | ||
| "version": "2", | ||
| "updated_at": "2026-01-01T00:00:00Z", | ||
| "skills": { | ||
| "test-stable": { | ||
| "version": "1.0.0", | ||
| "description": "Stable test skill", | ||
| "files": ["SKILL.md"], | ||
| "repo_dir": "skills" | ||
| } | ||
| } | ||
| } | ||
| ''' | ||
|
|
||
| [[Server]] | ||
| Pattern = "GET /test-ref/skills/test-stable/SKILL.md" | ||
| Response.Body = '''--- | ||
| name: test-stable | ||
| --- | ||
|
|
||
| # Test stable skill | ||
| ''' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nice to have] The success path, where the real
MustAnyClientruns on the shadow command and the copied config reaches root'sUpload, is covered only by a unit test with a fake resolver. If the shadow stopped seeing--profile, or root uploaded from a different context,aitools installwould silently stop uploading its event and no test would fail. Please add a case underacceptance/cmd/aitools/telemetry/, which already records/telemetry-extand hasprint_telemetry. It would runaitools install --skills-only --scope=global --agents=claude-codeagainst the mock server, with this test's manifest stubs, and print the uploadedaitools_install_event.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added
acceptance/cmd/aitools/telemetry/installin e503a62. It runsaitools install --skills-only --scope=global --agents=claude-codeagainst the mock server and prints the uploadedaitools_install_eventfor PAT auth and for adatabricks-cliprofile passed with--profileand a valid cached token. A third case uses an expired cached token and checks that nothing is refreshed or uploaded.While there: the new base includes #7003 (
titleadds its own trailing newline), so I dropped the\nfrom this script's titles, which had made the test fail on the restacked head.