Skip to content

air: Name the YAML path in unknown config field errors - #7526

Closed
abdulkareem-db wants to merge 2 commits into
databricks:mainfrom
abdulkareem-db:air-friendly-unknown-config-fields
Closed

abdulkareem-db wants to merge 2 commits into
databricks:mainfrom
abdulkareem-db:air-friendly-unknown-config-fields

Conversation

@abdulkareem-db

Copy link
Copy Markdown

Changes

Unknown or misspelled keys in an air run workload YAML now produce an error that names the YAML path and line, suggests the closest valid field, lists the valid fields at that level, and points at the matching -h config.<path> help. When a key has no close match, the error also says it may come from newer documentation than this CLI version supports.

Before:

Error: invalid config workload.yaml: yaml: unmarshal errors:
  line 6: field rot_path not found in type aircmd.codeSourceConfig

After:

Error: invalid config workload.yaml: line 6: unknown field "code_source.rot_path"; did you mean "root_path"?
fields under "config.code_source" are: git, include_paths, remote_volume, root_path
run "databricks air run -h config.code_source" for details

No close match:

Error: invalid config workload.yaml: line 7: unknown field "code_source.fancy_new_option"
fields under "config.code_source" are: git, include_paths, remote_volume, root_path
run "databricks air run -h config.code_source" for details

fields from newer documentation may not be supported by this CLI (version X.Y.Z); update the Databricks CLI (run 'databricks version --check')

Why

The go-yaml error exposes internal Go type names (aircmd.runConfig, aircmd.codeSourceConfig) and gives no path or suggestion. A common way to hit it is running a current workload YAML on an older CLI. For example, the flat code_source.root_path form is rejected by CLIs before v1.20.0 and nothing in the error suggests upgrading.

The check walks the parsed YAML against configSchema(), the same schema that --override path validation and -h config use. It reuses findConfigChild and closestConfigField, so hidden and legacy fields (code_source.type, code_source.snapshot) and inline fields are handled the same way. All unknown keys are reported, not just the first. KnownFields(true) stays as a backstop. --override errors are unchanged.

Tests

  • Unit tests in cmd/air/runconfig_test.go: a top-level typo, a nested typo, an unknown key with no close match (version hint), several unknown keys reported together, line numbers, and hidden legacy keys still loading.
  • go test ./cmd/air/... passes.
  • Manually, with a built binary and air run --dry-run: the three error cases above, and an unchanged valid workload config still validates.

This pull request and its description were written by Isaac.

Unknown or misspelled keys in an `air run` workload YAML surfaced the raw
go-yaml error, which names internal Go types (e.g. "field rot_path not found
in type aircmd.codeSourceConfig") and gives no path or suggestion.

Walk the parsed YAML against configSchema() before decoding and report every
unknown key with its line, dotted YAML path, a did-you-mean suggestion, the
valid sibling fields, and the matching `-h config.<path>` pointer. When a key
has no close match, note that it may come from newer documentation than the
running CLI version supports. KnownFields(true) stays as a backstop.

Fixes CNXT-2566 and CNXT-2554.

Co-authored-by: Isaac <no-reply@databricks.com>
@abdulkareem-db
abdulkareem-db requested review from a team as code owners November 11, 2026 23:53
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! This pull request was opened from a fork.

If you're a member of the Databricks organization, please request access to the cli-contrib team and open your pull request from a branch in this repository instead of a fork — fork PRs don't run integration tests automatically and have reduced CI access.

If you're an external contributor, no action is needed; a maintainer will review your pull request.

@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 11, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 7526
  • Commit SHA: 19e6703f2ce13a8e5350dc1da61ff33227404470

Checks will be approved automatically on success.

@abdulkareem-db

Copy link
Copy Markdown
Author

Superseded by #7528 (same change, opened from a branch in this repo).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant