Repository navigation
[air] Reorganize databricks air help around the run lifecycle - #7512
Conversation
|
Commit: Regression test reportSelected 3 changed tests (limit=20, 0 not selected) 3 tests: regression=3 unreleased=0 coverage=0 failing=0
TestAccept/air/config-help: branch ✅ | main (a0a6430) ❌ | latest (v1.26.0) ❌main (a0a6430): latest (v1.26.0): same output as main (a0a6430) TestAccept/air/help: branch ✅ | main (a0a6430) ❌ | latest (v1.26.0) ❌main (a0a6430): latest (v1.26.0): same output as main (a0a6430) TestAccept/help: branch ✅ | main (a0a6430) ❌ | latest (v1.26.0) ❌main (a0a6430): latest (v1.26.0): |
ben-hansen-db
left a comment
There was a problem hiding this comment.
This reorg is great, thanks for putting this up. We should fyi Maggie or Caroline
In your config.compute example it seems that A10 are missing, is that intentional?
| root_path: . | ||
| command: python "$CODE_SOURCE_PATH/train.py" | ||
|
|
||
| Required: |
There was a problem hiding this comment.
nice I like this organization
✅ Integration test reportCommit: 0415eb2
Top 6 slowest tests (at least 2 minutes):
|
5a7a473 to
a42b794
Compare
| Describe a workload in a YAML file, submit it with "air run", then follow it | ||
| with "air list", "air get", and "air logs". | ||
|
|
||
| Get started: |
There was a problem hiding this comment.
imo this feels a bit long to be including at the top of every --help
There was a problem hiding this comment.
agreed. just updated
caroline-db
left a comment
There was a problem hiding this comment.
overall looks good, left a couple small comments/suggestions. thanks for this!
| &cobra.Group{ID: runGroupID, Title: "Run workloads:"}, | ||
| &cobra.Group{ID: manageGroupID, Title: "Manage runs:"}, | ||
| &cobra.Group{ID: packageGroupID, Title: "Package and deploy:"}, | ||
| &cobra.Group{ID: capacityGroupID, Title: "Provisioned capacity (optional):"}, |
There was a problem hiding this comment.
I find the use of (optional) here to be confusing, since we aren't explicitly classifying other groups as required/optional. was the intent because it's not a GA feature or something else?
(I think the use of provisioned capacity (optional) inside the compute help where we did use (required) for accelerator_type/num_accelerators works there though)
| type environmentConfig struct { | ||
| Dependencies dependencies `yaml:"dependencies" help:"Inline list of packages to install. Not allowed alongside unity_catalog_image."` | ||
| Version stringOrInt `yaml:"version" help:"Client image version to pin (default: 6). See environment version release notes: https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/"` | ||
| Version stringOrInt `yaml:"version" help:"Managed environment to start from, e.g. 6 for the Standard environment or databricks_ai_v6 for the Databricks AI environment (default: 6). Standard includes CUDA but, from version 5, not PyTorch; add it with dependencies. The Databricks AI environment adds PyTorch, Transformers, and other ML libraries. Supported versions and package lists: https://docs.databricks.com/aws/en/machine-learning/ai-runtime/develop-workloads/environment"` |
There was a problem hiding this comment.
we should include the azure link here as well
| @@ -53,26 +53,26 @@ var secretTemplateRe = sync.OnceValue(func() *regexp.Regexp { return regexp.Must | |||
| // runConfig is the top-level run YAML schema: experiment_name + compute / | |||
| // environment / code_source plus the command and run options. | |||
| type runConfig struct { | |||
There was a problem hiding this comment.
I like introducing sections, it makes it clearer. here's another variant that I think is even easier to read, thoughts? It organizes by purpose instead of making required a separate section and advanced a catch-all, plus slight section renaming.
Workload:
experiment_name (required)
compute (required)
command (required)
environment
code_source
env_variables
secrets
parameters
containers
Execution:
max_retries
timeout_minutes
idempotency_token
MLflow tracking:
mlflow_run_name
mlflow_experiment_directory
mlflow_artifact_location
Access and billing:
permissions
usage_policy_name
usage_policy_id
There was a problem hiding this comment.
Makes sense! I switched billing to cost attribution as that seems to be more aligned with how we usually describe usage policy
Group `air` subcommands by task (run, manage, package, provisioned capacity), add a get-started example and a "Learn more" footer, add examples to `air run`, and list YAML fields under headings so required and common fields come before advanced and pool-only ones. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
`command` is required unless `containers` is set, and `compute.num_accelerators` / `compute.accelerator_type` are always required, matching what validation enforces. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Name the values that run (4-6, databricks_ai_v5, databricks_ai_v6), what the Standard and Databricks AI environments contain, and link the AI Runtime environment page instead of the general serverless notes. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
The supported set changes with each environment release; the linked page stays current. Co-authored-by: Isaac <no-reply@databricks.com>
It pushed the command list down; `air run -h config` already has a minimal example and the footer points to it. Co-authored-by: Isaac <no-reply@databricks.com>
A bare "air run" reads like a standalone binary. Also point to logs and get before list in the intro, since following a submitted run comes first. Co-authored-by: Isaac <no-reply@databricks.com>
Workload / Execution / MLflow tracking / Access and cost attribution, with required fields leading their group and marked inline instead of a separate Required heading. Drop "(optional)" from the `air -h` pools group; it stays in `config.compute` next to the required fields. Co-authored-by: Isaac <no-reply@databricks.com>
64cdd95 to
ffe2c54
Compare
A `docs` struct tag holds the page path shared by both clouds, so field help stays short and no longer links only to AWS. Co-authored-by: Isaac <no-reply@databricks.com>
| Workload YAML schema. For details on a field, pass its path to -h, for example: | ||
|
|
||
| Install the skill locally for your coding agent: | ||
| databricks air run -h config.compute.accelerator_type |
There was a problem hiding this comment.
I think this would be a better example here since this help only lists top level fields, it's slightly unclear where accelerator_type is coming from
| databricks air run -h config.compute.accelerator_type | |
| databricks air run -h config.compute |
❌ Integration test reportCommit: 87716ec
1083 interesting tests: 995 MISS, 80 FAIL, 8 RECOVERED
Top 50 slowest tests (at least 2 minutes):
|
## Changes Follow-up to databricks#7512: `databricks air -h` describes `pools` as optional. ```diff Provisioned capacity: - pools View GPU pools provisioned for your workspace + pools View provisioned GPU pools, if any (not required to run) ``` ## Why Few workspaces have GPU pools, and the old line read as something every workspace has or needs before running. ## Tests - Regenerated `acceptance/air/help`; all `air` acceptance tests and `acceptance/help` pass. - `./task checks` and lint pass. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
Changes
Reorganize the
databricks airhelp pages around the run lifecycle:air -h: subcommands are grouped under Run workloads, Manage runs, Package and deploy, and Provisioned capacity instead of one A→Z list. Adds a two-line intro and a "Learn more" footer (YAML fields and example, AWS/Azure quickstart, agent skill). Drops "under active development" (air: remove stale Public Preview label from CLI help #7502 already removed the "Public Preview" label).air run -h: adds Examples (--watch,--dry-run,--override) and lists flags most-used first.air run -h config: adds a minimal YAML example and lists fields by purpose under Workload / Execution / MLflow tracking / Access and cost attribution, using a newgroupstruct tag. Required fields come first in each group and are marked inline. The agent-skill instructions move below the field list, and theair -hfooter also points to the skill.air run -h config.compute:pool_idandpriority_classmove under "Provisioned capacity (optional)".commandshows "(required unless containers is set)",compute.num_accelerators/compute.accelerator_typeare marked required (they previously showedRequired: no), and conditional requirements show inline, e.g.code_source.root_path"(required when code_source is set)".environment.versionhelp gives examples of both forms (6,databricks_ai_v6), says what the Standard and Databricks AI environments contain, and links the AI Runtime environment page (which lists supported versions) instead of the general serverless release notes.docsstruct tag with the path both clouds share) instead of AWS-only inline links. Applies toenvironment.version,compute.num_accelerators, andcompute.accelerator_type.databricks air run -h config.compute.accelerator_type) instead of bare paths.YAML_PATH; shorterpools get), help text names full commands (databricks air run, notair run), and usage-policy help says "attribute the run's usage to" instead of "bill the run to", matching the usage-policy docs.Before / after
Taken from the acceptance golden files; the GPU list in
config.computecomes from the test stub. Global Flags are unchanged and omitted.databricks air -hBefore
After
databricks air run -hBefore
After
databricks air run -h configBefore
After
databricks air run -h config.computeBefore
After
databricks air run -h config.environment.versionBefore
After
Why
New users read
poolsandpool_idas prerequisites because they sit next torunandaccelerator_type. The top-level help also listed commands A→Z, soruncame last, and the YAML field list was one flat list.Tests
air -h, config field headings and order, required-first ordering, the inline required markers, and the per-cloud Docs section (plus a check that no field help inlines a docs link).acceptance/air/help,acceptance/air/config-help, andacceptance/helpoutputs../task fmt,./task checks,./task lintpass.environment.versionvalues with real runs:4,5,6,databricks_ai_v5, anddatabricks_ai_v6run;1-3fail at runtime ("client image version must be between 4 and 6"), and7anddatabricks_ai_v7are rejected at submit.--dry-runwith the matching error, a run with onlyexperiment_name,compute, andcommandsucceeds, and both YAML examples shown in the help run successfully.This pull request and its description were written by Isaac.