Skip to content

air: list a non-git code_source root_path from root_path, not the working directory - #7646

Open
dimk0 wants to merge 2 commits into
mainfrom
cnxt-2663-non-git-root-listing
Open

dimk0 wants to merge 2 commits into
mainfrom
cnxt-2663-non-git-root-listing

Conversation

@dimk0

@dimk0 dimk0 commented Nov 16, 2026

Copy link
Copy Markdown
Member

Changes

air run lists the files of a code_source.root_path that isn't in a git repository by running git ls-files against a temporary git directory. That call now passes -C <root_path>, as the call for git repositories already does, so the listing no longer depends on the directory the CLI runs from.

Follow-up to #7643, which notes this in its description. The two changes are independent and can merge in either order.

Why

Without -C, git resolves include_paths and prints file names relative to the working directory. Run from a subdirectory of root_path, for example the workload's own directory with root_path: .., the snapshot silently drops files or packs a same-named file from root_path instead:

  • With include_paths: [ray_bootstrap.sh, train], run from train/, v1.26.0 uploads an empty snapshot. The run then fails on the node with Could not determine top-level component from tarball. On main, the empty-snapshot check rejects the upload instead, with an error that names paths that exist.
  • Without include_paths, run from train/, only train/'s files are listed, and each is read from root_path: the snapshot holds the root's README.md instead of train/README.md.

This affects copies of a project that aren't git checkouts, such as a downloaded archive of the AI Runtime cookbook, whose getting-started Ray recipes use root_path: .. and are run from their own directories. Git checkouts already list relative to root_path.

Tests

  • TestSnapshotFiles_NonGitRootFromSubdirectory runs from root/sub, where sub/main.py shadows main.py. With include_paths: [main.py, data] it now selects data/x.csv and main.py (before: main.py only); without include_paths it selects all three files (before: main.py only).
  • acceptance/air/run adds a dry run from a subdirectory of a non-git root_path, which reports 2 files (before: 1).
  • Both fail without the fix. go test ./cmd/air/... and go test ./acceptance -run TestAccept/air pass, and golangci-lint reports no issues on cmd/air. TestLogsCommandRepeatsRunLinkAfterStreaming needs DATABRICKS_WORKSPACE_ID unset locally, as on main.

This pull request and its description were written by Isaac.

dimk0 and others added 2 commits November 16, 2026 00:46
…king directory

For a root_path outside any git repository, snapshotFiles ran git ls-files
without -C, so git resolved include_paths and printed file names relative
to the CLI's working directory. Run from a subdirectory of root_path, such
as the workload's own directory with root_path .., the snapshot silently
dropped files or packed same-named files from root_path, and could be
empty. Pass -C repoPath, as the git-repository listing already does.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

Commit: 506378e99 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0
test branch main (20fef7c) latest (v1.26.0)
TestAccept/air/run ✅ ❌ ❌
TestAccept/air/run: branch ✅ | main (20fef7c) ❌ | latest (v1.26.0) ❌

main (20fef7c):

acceptance_test.go:1374: Diff:
        --- air/run/output.txt
        +++ /tmp/TestAcceptairrun3104221730/002/output.txt
        @@ -68,7 +68,7 @@
         === a non-git root_path selects the same files when run from its subdirectory
         
         >>> [CLI] air run -f workload.yaml --dry-run
        -Dry run: code_source snapshot would include 2 file(s).
        +Dry run: code_source snapshot would include 1 file(s).
         Dry run: configuration for "smoke-test" is valid; not submitting.
         
         === git.remote is rejected
        
FAIL

latest (v1.26.0):

acceptance_test.go:1374: Diff:
        --- air/run/output.txt
        +++ /tmp/TestAcceptairrun3092261101/002/output.txt
        @@ -52,23 +52,16 @@
         === code_source config passes validation and reports the selected files
         
         >>> [CLI] air run -f with-code-source.yaml --dry-run
        -Dry run: code_source snapshot would include 1 file(s).
         Dry run: configuration for "smoke-test" is valid; not submitting.
         
         === git-ignored root_path is rejected instead of selecting no files
         
         >>> [CLI] air run -f ignored-root.yaml --dry-run
        -Error: code_source snapshot of [TEST_TMP_DIR]/.scratch/proj contains no files; refusing to upload an empty snapshot: root_path is ignored by git (.gitignore:1:.scratch/), and the snapshot only includes files git does not ignore. Move the project out of the ignored directory, un-ignore it, or change code_source.root_path. Inspect the rule with `git -C [TEST_TMP_DIR]/.scratch/proj check-ignore -v .`
        +Dry run: configuration for "smoke-test" is valid; not submitting.
         
        -Tip: use the --debug flag to see more details and a trace of this error:
        -  databricks --debug air run …
        -
        -Exit code: 1
        -
         === a non-git root_path selects the same files when run from its subdirectory
         
         >>> [CLI] air run -f workload.yaml --dry-run
        -Dry run: code_source snapshot would include 2 file(s).
         Dry run: configuration for "smoke-test" is valid; not submitting.
         
         === git.remote is rejected
        
FAIL

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 506378e

Run: 43782613194

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 16 3:48
💚​ aws-windows-latest-4core-16gb 1 272 14 3:13
💚​ azure linux-2core-8gb 1 269 16 3:52
💚​ azure-windows-latest-4core-16gb 1 271 14 2:53
💚​ gcp linux-2core-8gb 1 270 16 3:34
💚​ gcp-windows-latest-4core-16gb 1 272 14 3:26
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:21 gcp-windows-latest-4core-16gb TestAccept
3:08 aws-windows-latest-4core-16gb TestAccept
2:49 azure-windows-latest-4core-16gb TestAccept

@dimk0
dimk0 marked this pull request as ready for review November 16, 2026 01:16
@dimk0
dimk0 requested review from a team as code owners November 16, 2026 01:16
@dimk0
dimk0 requested a review from shreyas-goenka November 16, 2026 01:16

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants