Repository navigation
Conversation
For a working-tree snapshot, git ls-files silently ignores a pathspec that matches nothing, so a mistyped or stale include_paths entry was left out of the snapshot. The run then failed on the node after compute was provisioned. Commit-pinned snapshots already reject missing include_paths; working-tree snapshots now do the same before packaging. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
- Skip the unmatched-entry check for pathspec magic (":!x", ":(exclude)x",
":/x") and backslash escapes, as for globs. These are patterns, and an
exclusion can never select a file, so the check rejected valid configs.
- Drop entries that the tarball writer does not pack (directories such as a
nested git repository or a submodule) from the listing, so the empty and
unmatched checks see what is uploaded.
- Reject listed files outside root_path, which pathspec magic such as
":(top)x" can select.
- Reword the error: it lists only the unmatched entries, and empty
directories, nested repositories and submodules add no files.
- Cover the "." shortcut, the path-component boundary, globs, exclusions,
nested repositories and the containment check in tests.
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
✅ Integration test reportCommit: c9dec55
Top 3 slowest tests (at least 2 minutes):
|
… repositories - Only suggest `git check-ignore -v` when root_path is in a git repository; outside one the command fails. - The empty-snapshot errors now say that nested git repositories and submodules add no files, since such entries no longer count as files. - Test a backslash-escaped entry, which git reads as a pattern, and the hint outside git. Co-authored-by: Isaac <no-reply@databricks.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
air run: for a working-tree snapshot (nocode_source.git), rejectcode_source.include_pathsentries that select no files, and name them, before packaging. Commit-pinned snapshots already reject missing include paths. Glob patterns, pathspec magic (:!x,:(exclude)x,:/x) and backslash escapes are left to git, which matches them as patterns.root_path, which pathspec magic such as:(top)xcan select.git check-ignore -vonly whenroot_pathis in a git repository.Why
git ls-filesignores a pathspec that matches nothing, so a mistyped or stale entry was dropped from the snapshot without a warning,--dry-runpassed, and the run failed on the node after compute was provisioned. A realistic case is the AI Runtime cookbook's getting-started Ray recipes, which list the shared bootstrap and the recipe directory ininclude_paths:include_paths: [ray_bootstrap.sh, my_job]for amy-jobdirectory uploaded onlyray_bootstrap.sh.Replaces #7639, opened from a fork where CI could not get its OIDC token.
Tests
TestSnapshotFiles_IncludePathMatchingaccepts.,./, a file,?and[...]globs, and:!/:(exclude)exclusions with the expected files, and rejects[data_v2, data]naming onlydata.TestSnapshotFiles_NestedRepoSelectsNoFilesandTestSnapshotFiles_RejectsPathspecOutsideRootcover nested repositories and the containment check;TestSnapshotFiles_AcceptsBackslashEscapedIncludePath(not on Windows) andTestSnapshotFiles_OmitsIgnoreRuleHintOutsideGitcover backslash escapes and the hint. The earlier rejection and accepted-forms tests still pass..shortcut, the path-component boundary, the glob, magic and backslash skips, the nested-repository skip, containment) fails its test when removed.go test ./cmd/air/...andgo test ./acceptance -run TestAccept/airpass, andgolangci-lint run ./cmd/air/...reports no issues. LocallyTestLogsCommandRepeatsRunLinkAfterStreamingfails onmaintoo whenDATABRICKS_WORKSPACE_IDis exported; it passes with the variable unset.root_path,ls-filesruns without-C, so the listing depends on the working directory. That predates this change and is left for a follow-up.This pull request and its description were written by Isaac.