Skip to content

[draft] Accept YAML arrays for for_each_task inputs - #7304

Open
lennartkats-db wants to merge 8 commits into
mainfrom
factory/dabs/d22b087ea1b2
Open

lennartkats-db wants to merge 8 commits into
mainfrom
factory/dabs/d22b087ea1b2

Conversation

@lennartkats-db

@lennartkats-db lennartkats-db commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Changes

  • Accept literal YAML arrays written directly in job for_each_task.inputs, including target overrides, and send their JSON encoding in the Jobs API string field.
  • Preserve existing string inputs, including their formatting and interpolation. Reject unescaped ${...} references inside YAML arrays with an error naming the reference, path, and YAML location; use the JSON string form for interpolated inputs.
  • Allow strings or arrays in the bundle JSON schema. PyDABs continues to model this SDK field as a string because its resource values do not pass through bundle YAML loading.
  • Add a changelog fragment and acceptance coverage for validation, Jobs API create requests, target precedence, existing string interpolation, and JSON string inputs in variable-defined task containers.

Scope

This PR supports arrays authored directly at for_each_task.inputs. It does not serialize array-valued complex variables such as inputs: ${var.items}, or arrays inside variable-defined task or job containers. Issue #6901 tracks that separate variable-resolution behavior. The variable-container fixture here checks that existing JSON string inputs still work through those containers.

Why

Issue #2370 reports that bundle authors must hand-escape JSON arrays to configure for-each task inputs. The SDK models this field as a string, so typed normalization cannot accept a native YAML sequence. Encoding directly authored arrays during loading supports the requested syntax while retaining existing string behavior.

Fixes #2370

Tests

  • go test ./bundle/config ./bundle/schema ./bundle/internal/schema -count=1
  • go test ./acceptance -run '^TestAccept/bundle/resources/jobs/for_each_inputs' -test.v -timeout=20m against the local fake server, with both DMS modes. The fixture records the Jobs API create payloads.
  • Directly invoked the four Python codegen schema test functions and generated PyDABs output in a temporary directory. uv run --frozen -m pytest could not install its dependencies locally because the PyPI connection reset while fetching pygments.
  • The deployment fixture is enabled for cloud CI. I did not run a live workspace deployment locally.

@lennartkats-db lennartkats-db added DABs DABs related issues software-factory PR produced by the software factories task labels Nov 3, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 2e23b88

Run: 43446913855

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 273 17 8:24
🔄​ aws-windows-latest-4core-16gb 1 1 274 15 6:43
💚​ azure linux-2core-8gb 1 272 17 9:17
💚​ azure-windows-latest-4core-16gb 1 274 15 5:17
🔄​ gcp linux-2core-8gb 1 1 272 17 10:30
💚​ gcp-windows-latest-4core-16gb 1 275 15 5:31
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncEnsureRemotePathIsUsableIfRepoExists ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestSyncFullFileSync ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:17 azure-windows-latest-4core-16gb TestAccept
3:08 gcp-windows-latest-4core-16gb TestAccept
3:04 aws-windows-latest-4core-16gb TestAccept
2:36 aws-windows-latest-4core-16gb TestAccept/bundle/resources/jobs/for_each_inputs/literals/DMS=
2:26 azure linux-2core-8gb TestImportDirDoesNotOverwrite
2:23 gcp linux-2core-8gb TestAccept/bundle/resources/jobs/for_each_inputs/literals/DMS=true

@lennartkats-db lennartkats-db added software-factory PR produced by the software factories task and removed software-factory PR produced by the software factories task labels Nov 3, 2026
@lennartkats-db
lennartkats-db marked this pull request as ready for review November 4, 2026 19:21
@lennartkats-db
lennartkats-db requested review from a team as code owners November 4, 2026 19:21
@lennartkats-db
lennartkats-db marked this pull request as draft November 4, 2026 19:21
@lennartkats-db
lennartkats-db marked this pull request as ready for review November 4, 2026 22:00
@lennartkats-db lennartkats-db changed the title [draft] Fix bundle/config/root.go [draft] Accept YAML arrays for for_each_task inputs Nov 5, 2026

@andrewnester andrewnester left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few cross-cutting asks:

  • bundle/schema/jsonschema.json still types for_each_task.inputs as $defs/string, so the YAML array the CLI now accepts is flagged as invalid by schema-aware editors. The shadowed definition / serialized_dashboard fields get $defs/interface from their Go type; here the Go type can't change, so the schema needs an explicit string-or-array. Can a per-type hook in bundle/internal/schema/main.go (like removeJobsFields, keyed on reflect.TypeFor[jobs.ForEachTask]()) add that, with the schema regenerated?
  • This is user-visible config behaviour with only unit coverage. acceptance/bundle/resources/cluster_policies/definition_yaml is the template for the same feature on cluster policies: validate -o json, deploy, print_requests.py on the create call, Cloud = true. The description says no live workspace run was done, and the array-of-objects case in particular relies on the Jobs API accepting that JSON. Can you add an equivalent test under acceptance/bundle/resources/jobs/ with Cloud = true, including the target-override and variable-reference cases?
  • No .nextchanges/bundles/ fragment; this is user-visible, so it needs one per .nextchanges/README.md.

Comment thread bundle/config/root.go Outdated
return dyn.InvalidValue, fmt.Errorf("%s at %s: %w", p, input.Location(), err)
}
// Leave unresolved references for normal validation instead of interpolating into JSON text.
if dynvar.ContainsVariableReference(string(data)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With inputs: ["${var.a}_x", "y"] and a plain string variable a, this branch hands the sequence back, convert.Normalize then warns expected string, found sequence and required field "inputs" is not set, and bundle validate exits 0 with inputs gone (checked against the fake server). The hand-written JSON string form with the same ${var.a} inside interpolates fine via ResolveVariableReferencesOnlyResources, so the array form is the only one that can't carry a reference, and the warning never mentions the reference. Can we marshal regardless and let the usual string interpolation run on the JSON text, as the string form already does, or else fail here with an error that names the reference instead of falling through to a type warning?

@lennartkats-db lennartkats-db Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this to fail during loading when a YAML array contains an unescaped ${...} reference. The error names the reference and YAML location, so bundle validate exits nonzero. The array_reference acceptance fixture asserts that behavior. I kept string inputs for interpolation because substituting a quoted value after JSON encoding could change the payload.

-- Isaac

Comment thread bundle/config/root.go
}))
}

func rewriteForEachTaskInputs(v dyn.Value) (dyn.Value, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConfigureClusterPolicyDefinition and ConfigureDashboardSerializedDashboard in bundle/config/mutator/resourcemutator/ already do this job (inline YAML to a JSON string for an SDK string field) as resource mutators after variable resolution, with an error diagnostic for an unsupported kind. This one runs in the loader before normalization. I assume that is because jobs.Task.ForEachTask.Inputs can't be shadowed with any the way resources.ClusterPolicy.Definition is; can the doc comment on this function say that, and spell out how its behaviour differs from those mutators (timing relative to variables, no error on other kinds)?

@lennartkats-db lennartkats-db Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expanded the loader comment to explain that a resource-level any shadow cannot cover this nested SDK field, so literal arrays are encoded before typed normalization. The cluster policy and dashboard mutators run after variable resolution. Other kinds still use normal validation.

-- Isaac

Comment thread bundle/config/root.go Outdated
})
}

func isForEachTaskInputsPath(p dyn.Path) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bundle/config/validate/single_node_cluster.go:117-119 matches this same task path with dyn.NewPattern(..., dyn.Key("tasks"), dyn.AnyIndex(), dyn.Key("for_each_task"), ...) and dyn.MapByPattern. Can this use patterns for the three prefixes (root, targets.*, environments.*) instead of the length and Key() != "" checks, so the path shape lives in the idiom the rest of the repo uses?

@lennartkats-db lennartkats-db Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced the length and key checks with dyn.Pattern matches for root, targets.*, and environments.* paths. The root and target cases exercise the matched task path.

-- Isaac

Comment thread bundle/config/root.go Outdated
return input, nil
}

data, err := json.Marshal(input.AsAny())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AsAny() turns a YAML timestamp into time.Time, so inputs: [2024-01-01, 2024-01-02] is sent as ["2024-01-01T00:00:00Z","2024-01-02T00:00:00Z"], while the same scalar outside an array keeps its literal through normalizeString (MustTime().String()). Is the reformat intended, or should time values be stringified the way normalize does before marshalling?

@lennartkats-db lennartkats-db Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I now convert dyn.KindTime nodes to their original MustTime().String() values before marshaling. The unit test covers date-only elements and a nested timestamp with fractional seconds and a timezone offset.

-- Isaac

{name: "YAML object array", inputs: "[{name: input_1, enabled: true}]", want: `[{"enabled":true,"name":"input_1"}]`},
{name: "empty YAML array", inputs: "[]", want: `[]`},
{name: "JSON string", inputs: `'[ "input_1", "input_2" ]'`, want: `[ "input_1", "input_2" ]`},
{name: "variable reference string", inputs: `${var.items}`, want: `${var.items}`},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This case is a string, so it never reaches the sequence branch in rewriteForEachTaskInputs; the array-with-reference path (["${var.a}_x", "y"]) is untested, and today it would fail this test: inputs ends up empty with only a warning. Can you add that case so the expected behaviour for references inside the array is pinned down?

@lennartkats-db lennartkats-db Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added tests for array references in a root element, a nested object, and a target override. The acceptance fixture confirms that bundle validate exits nonzero and names the reference, path, and source location. I also checked that an embedded reference in the existing JSON string form still reaches the Jobs create request with its interpolated value.

-- Isaac

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 2e23b883d | Workflow run

Regression test report

Selected 3 changed tests (limit=20, 0 not selected)

6 tests: regression=4 unreleased=0 coverage=2 failing=0
test branch main (8e70c8b) latest (v1.26.0)
TestAccept/bundle/resources/jobs/for_each_inputs/array_reference/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/jobs/for_each_inputs/array_reference/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/jobs/for_each_inputs/literals/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/jobs/for_each_inputs/literals/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/jobs/for_each_inputs/variable_containers/DMS= ✅ ✅ ✅
TestAccept/bundle/resources/jobs/for_each_inputs/variable_containers/DMS=true ✅ ✅ ✅
TestAccept/bundle/resources/jobs/for_each_inputs/array_reference/DMS=: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate\nError: failed to rewrite for_each_task.inputs in [TEST_TMP_DIR]/databricks.yml: resources.jobs.fan.tasks[0].for_each_task.inputs at [TEST_TMP_DIR]/databricks.yml:15:15 contains variable reference ${var.item} in a YAML array; use a JSON string for interpolated inputs\n\n\nFound 1 error\n"
        	            	actual  : "\n>>> [CLI] bundle validate\nWarning: expected string, found sequence\n  at resources.jobs.fan.tasks[0].for_each_task.inputs\n  in databricks.yml:15:15\n\nWarning: required field \"inputs\" is not set\n  at resources.jobs.fan.tasks[0].for_each_task\n  in databricks.yml:14:13\n\nName: for_each_inputs_array_reference\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/for_each_inputs_array_reference/default\n\nFound 2 warnings\n\nUnexpected success\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -2,6 +2,21 @@
        	            	 >>> [CLI] bundle validate
        	            	-Error: failed to rewrite for_each_task.inputs in [TEST_TMP_DIR]/databricks.yml: resources.jobs.fan.tasks[0].for_each_task.inputs at [TEST_TMP_DIR]/databricks.yml:15:15 contains variable reference ${var.item} in a YAML array; use a JSON string for interpolated inputs
        	            	+Warning: expected string, found sequence
        	            	+  at resources.jobs.fan.tasks[0].for_each_task.inputs
        	            	+  in databricks.yml:15:15
        	            	 
        	            	+Warning: required field "inputs" is not set
        	            	+  at resources.jobs.fan.tasks[0].for_each_task
        	            	+  in databricks.yml:14:13
        	            	 
        	            	-Found 1 error
        	            	+Name: for_each_inputs_array_reference
        	            	+Target: default
        	            	+Workspace:
        	            	+  User: [USERNAME]
        	            	+  Path: /Workspace/Users/[USERNAME]/.bundle/for_each_inputs_array_reference/default
        	            	 
        	            	+Found 2 warnings
        	            	+
        	            	+Unexpected success
        	            	+
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/resources/jobs/for_each_inputs/array_reference/DMS=
        	Messages:   	bundle/resources/jobs/for_each_inputs/array_reference/output.txt vs /tmp/TestAcceptbundleresourcesjobsfor_each_inputsarray_reference1601308328/002/output.txt
    acceptance_test.go:1164: Test produced unexpected files:
        .databricks/.gitignore

latest (v1.26.0): same output as main (8e70c8b)

TestAccept/bundle/resources/jobs/for_each_inputs/array_reference/DMS=true: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate\nError: failed to rewrite for_each_task.inputs in [TEST_TMP_DIR]/databricks.yml: resources.jobs.fan.tasks[0].for_each_task.inputs at [TEST_TMP_DIR]/databricks.yml:15:15 contains variable reference ${var.item} in a YAML array; use a JSON string for interpolated inputs\n\n\nFound 1 error\n"
        	            	actual  : "\n>>> [CLI] bundle validate\nWarning: expected string, found sequence\n  at resources.jobs.fan.tasks[0].for_each_task.inputs\n  in databricks.yml:15:15\n\nWarning: required field \"inputs\" is not set\n  at resources.jobs.fan.tasks[0].for_each_task\n  in databricks.yml:14:13\n\nName: for_each_inputs_array_reference\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/for_each_inputs_array_reference/default\n\nFound 2 warnings\n\nUnexpected success\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -2,6 +2,21 @@
        	            	 >>> [CLI] bundle validate
        	            	-Error: failed to rewrite for_each_task.inputs in [TEST_TMP_DIR]/databricks.yml: resources.jobs.fan.tasks[0].for_each_task.inputs at [TEST_TMP_DIR]/databricks.yml:15:15 contains variable reference ${var.item} in a YAML array; use a JSON string for interpolated inputs
        	            	+Warning: expected string, found sequence
        	            	+  at resources.jobs.fan.tasks[0].for_each_task.inputs
        	            	+  in databricks.yml:15:15
        	            	 
        	            	+Warning: required field "inputs" is not set
        	            	+  at resources.jobs.fan.tasks[0].for_each_task
        	            	+  in databricks.yml:14:13
        	            	 
        	            	-Found 1 error
        	            	+Name: for_each_inputs_array_reference
        	            	+Target: default
        	            	+Workspace:
        	            	+  User: [USERNAME]
        	            	+  Path: /Workspace/Users/[USERNAME]/.bundle/for_each_inputs_array_reference/default
        	            	 
        	            	+Found 2 warnings
        	            	+
        	            	+Unexpected success
        	            	+
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/resources/jobs/for_each_inputs/array_reference/DMS=true
        	Messages:   	bundle/resources/jobs/for_each_inputs/array_reference/output.txt vs /tmp/TestAcceptbundleresourcesjobsfor_each_inputsarray_reference2608781666/002/output.txt
    acceptance_test.go:1164: Test produced unexpected files:
        .databricks/.gitignore

latest (v1.26.0): same output as main (8e70c8b)

TestAccept/bundle/resources/jobs/for_each_inputs/literals/DMS=: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: Diff:
        --- bundle/resources/jobs/for_each_inputs/literals/output.txt
        +++ /tmp/TestAcceptbundleresourcesjobsfor_each_inputsliteralsDMS=2138568992/002/output.txt
        @@ -1,13 +1,33 @@
         
         >>> [CLI] bundle validate -o json
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         [
           {
             "task_key": "array",
        -    "inputs": "[\"root_first\",\"root_second\"]"
        +    "inputs": null
           },
           {
             "task_key": "objects",
        -    "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]"
        +    "inputs": null
           },
           {
             "task_key": "string_interpolation",
        @@ -20,6 +34,26 @@
         ]
         
         >>> [CLI] bundle deploy
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/default/files...
         Created jobs.fan
         Files: 3 uploaded, 0 deleted
        @@ -44,7 +78,7 @@
             "tasks": [
               {
                 "for_each_task": {
        -          "inputs": "[\"root_first\",\"root_second\"]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/default/files/notebook",
        @@ -57,7 +91,7 @@
               },
               {
                 "for_each_task": {
        -          "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/default/files/notebook",
        @@ -99,14 +133,35 @@
         }
         
         >>> [CLI] bundle validate -o json -t override
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +     databricks.yml:67:17
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         [
           {
             "task_key": "array",
        -    "inputs": "[\"target_only\"]"
        +    "inputs": null
           },
           {
             "task_key": "objects",
        -    "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]"
        +    "inputs": null
           },
           {
             "task_key": "string_interpolation",
        @@ -119,6 +168,27 @@
         ]
         
         >>> [CLI] bundle deploy -t override
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +     databricks.yml:67:17
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/override/files...
         Created jobs.fan
         Files: 3 uploaded, 0 deleted
        @@ -143,7 +213,7 @@
             "tasks": [
               {
                 "for_each_task": {
        -          "inputs": "[\"target_only\"]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/override/files/notebook",
        @@ -156,7 +226,7 @@
               },
               {
                 "for_each_task": {
        -          "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/override/files/notebook",
        @@ -198,6 +268,27 @@
         }
         
         >>> [CLI] bundle destroy --auto-approve -t override
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +     databricks.yml:67:17
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         The following resources will be deleted:
           delete resources.jobs.fan
         
        @@ -206,6 +297,26 @@
         Destroy: 1 deleted
         
         >>> [CLI] bundle destroy --auto-approve
... (22 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.26.0): same output as main (8e70c8b)

TestAccept/bundle/resources/jobs/for_each_inputs/literals/DMS=true: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: Diff:
        --- bundle/resources/jobs/for_each_inputs/literals/output.txt
        +++ /tmp/TestAcceptbundleresourcesjobsfor_each_inputsliteralsDMS=tr4277907231/002/output.txt
        @@ -1,13 +1,33 @@
         
         >>> [CLI] bundle validate -o json
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         [
           {
             "task_key": "array",
        -    "inputs": "[\"root_first\",\"root_second\"]"
        +    "inputs": null
           },
           {
             "task_key": "objects",
        -    "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]"
        +    "inputs": null
           },
           {
             "task_key": "string_interpolation",
        @@ -20,6 +34,26 @@
         ]
         
         >>> [CLI] bundle deploy
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/default/files...
         Created jobs.fan
         Files: 3 uploaded, 0 deleted
        @@ -44,7 +78,7 @@
             "tasks": [
               {
                 "for_each_task": {
        -          "inputs": "[\"root_first\",\"root_second\"]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/default/files/notebook",
        @@ -57,7 +91,7 @@
               },
               {
                 "for_each_task": {
        -          "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/default/files/notebook",
        @@ -99,14 +133,35 @@
         }
         
         >>> [CLI] bundle validate -o json -t override
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +     databricks.yml:67:17
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         [
           {
             "task_key": "array",
        -    "inputs": "[\"target_only\"]"
        +    "inputs": null
           },
           {
             "task_key": "objects",
        -    "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]"
        +    "inputs": null
           },
           {
             "task_key": "string_interpolation",
        @@ -119,6 +168,27 @@
         ]
         
         >>> [CLI] bundle deploy -t override
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +     databricks.yml:67:17
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/override/files...
         Created jobs.fan
         Files: 3 uploaded, 0 deleted
        @@ -143,7 +213,7 @@
             "tasks": [
               {
                 "for_each_task": {
        -          "inputs": "[\"target_only\"]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/override/files/notebook",
        @@ -156,7 +226,7 @@
               },
               {
                 "for_each_task": {
        -          "inputs": "[{\"count\":3,\"enabled\":true,\"name\":\"quoted \\\"name\\\" with \\\\backslash\"},{\"count\":0,\"enabled\":false,\"name\":\"second\"}]",
        +          "inputs": "",
                   "task": {
                     "notebook_task": {
                       "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/override/files/notebook",
        @@ -198,6 +268,27 @@
         }
         
         >>> [CLI] bundle destroy --auto-approve -t override
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:20:21
        +
        +Warning: expected string, found sequence
        +  at resources.jobs.fan.tasks[1].for_each_task.inputs
        +  in databricks.yml:29:15
        +
        +Warning: expected string, found sequence
        +  at targets.override.resources.jobs.fan.tasks[0].for_each_task.inputs
        +  in databricks.yml:67:25
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[0].for_each_task
        +  in databricks.yml:20:13
        +     databricks.yml:67:17
        +
        +Warning: required field "inputs" is not set
        +  at resources.jobs.fan.tasks[1].for_each_task
        +  in databricks.yml:28:13
        +
         The following resources will be deleted:
           delete resources.jobs.fan
         
        @@ -206,6 +297,26 @@
         Destroy: 1 deleted
         
         >>> [CLI] bundle destroy --auto-approve
... (25 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.26.0): same output as main (8e70c8b)

@lennartkats-db lennartkats-db added the ai-auto PRs made by AI loops & factories, label Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto PRs made by AI loops & factories, DABs DABs related issues PyDABs software-factory PR produced by the software factories task

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DAB syntax for for_each task inputs is cumbersome

3 participants