Skip to content

Build a JUnit XML report from the results of a test run - #7299

Open
tharunbhuk-db wants to merge 1 commit into
tharun/pipelines-test-a4-run-controlfrom
tharun/pipelines-test-pr3-junit
Open

tharunbhuk-db wants to merge 1 commit into
tharun/pipelines-test-a4-run-controlfrom
tharun/pipelines-test-pr3-junit

Conversation

@tharunbhuk-db

@tharunbhuk-db tharunbhuk-db commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

🥞 Stacked PR


  • Build a JUnit XML report from a test run: one entry per case, plus a failing entry for an incomplete run.
  • The file is replaced atomically.
  • Not used by the command yet.

Tests: unit tests.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Commit: c7628c962 | Workflow run

Regression test report

Selected 13 changed tests (limit=20, 0 not selected)

13 tests: regression=13 unreleased=0 coverage=0 failing=0
test branch main (688c228) latest (v1.25.0)
TestAccept/pipelines/databricks-cli-help ✅ ❌ ❌
TestAccept/pipelines/test/arguments ✅ ❌ ❌
TestAccept/pipelines/test/immutable ✅ ❌ ❌
TestAccept/pipelines/test/pipeline-id ✅ ❌ ❌
TestAccept/pipelines/test/results/failed ✅ ❌ ❌
TestAccept/pipelines/test/results/incomplete ✅ ❌ ❌
TestAccept/pipelines/test/results/passed ✅ ❌ ❌
TestAccept/pipelines/test/run-control/canceled ✅ ❌ ❌
TestAccept/pipelines/test/run-control/not-test-update ✅ ❌ ❌
TestAccept/pipelines/test/run-control/resume ✅ ❌ ❌
TestAccept/pipelines/test/run-control/timeout ✅ ❌ ❌
TestAccept/pipelines/test/selection ✅ ❌ ❌
TestAccept/pipelines/test/submit-error ✅ ❌ ❌
TestAccept/pipelines/databricks-cli-help: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/databricks-cli-help/output.txt
        +++ /tmp/TestAcceptpipelinesdatabricks-cli-help2570642892/002/output.txt
        @@ -30,7 +30,6 @@
           run                   Run a pipeline
           show                  Preview a table's columns and sample rows
           stop                  Stop a pipeline
        -  test                  *Preview* Run the tests of a pipeline
         
         Management Commands
           apply-environment     *Public Preview* Apply the latest environment to the pipeline.

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/arguments: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments2547969729/002/output.txt
        @@ -1,97 +1,747 @@
         
         >>> [CLI] pipelines test --help
        -Run the tests of a pipeline in a test-only update.
        +The Lakeflow Spark Declarative Pipelines API allows you to create, edit,
        +  delete, start, and view details about pipelines.
         
        -This command is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it.
        +  Spark Declarative Pipelines is a framework for building reliable,
        +  maintainable, and testable data processing pipelines. You define the
        +  transformations to perform on your data, and Spark Declarative Pipelines
        +  manages task orchestration, cluster management, monitoring, data quality, and
        +  error handling.
         
        -KEY is the unique name of the pipeline, as defined in its YAML file.
        -It is required when test selections are given. If there is only one pipeline
        -in the project and no arguments are given, it is selected automatically.
        -Outside a project, identify the pipeline with --pipeline-id; all positional
        -arguments are then test selections.
        +  Instead of defining your data pipelines using a series of separate Apache
        +  Spark tasks, Spark Declarative Pipelines manages how your data is transformed
        +  based on a target schema you define for each processing step. You can also
        +  enforce data quality with Spark Declarative Pipelines expectations.
        +  Expectations allow you to define expected data quality and specify how to
        +  handle records that fail those expectations.
         
        -TEST_SELECTION is a test directory, file, or pytest test ID, for example
        -tests/test_orders.py::test_backfill. In a project, a relative path is a local
        -path, relative to the working directory, and is converted to its deployed
        -workspace path. With --pipeline-id, a relative path is relative to the
        -pipeline's root_path. Paths that start with / are workspace paths. Without a
        -selection, all of the pipeline's tests run.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Each --pytest-arg passes one argument to pytest, in the order given. Only -k
        -and -m are supported, each followed by its value in the next --pytest-arg, for
        -example:
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        -  databricks pipelines test my_pipeline --pytest-arg=-k --pytest-arg='orders and not slow'
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -The command waits for the results. It exits with code 0 only if at least one
        -test was collected and none failed or errored, so a run whose tests were all
        -skipped succeeds; otherwise it exits with code 1. --no-wait returns after the
        -update starts.
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        ---timeout and Ctrl+C stop waiting but not the update; pass its ID to --resume
        -to read the results later. To stop the update, run 'pipelines stop'.
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +=== The command is experimental and off by default
        +>>> musterr [CLI] pipelines test --pipeline-id 1234
        +Error: unknown flag: --pipeline-id
        +
         Usage:
        -  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Pipeline Test Flags:
        -      --no-wait                  Don't wait for the tests to complete.
        -      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        -      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        -      --resume string            Read the results of an existing test update instead of starting one.
        -      --timeout duration         Stop waiting for results after this duration, for example 30m. The update keeps running.
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
         Flags:
        -  -h, --help   help for test
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
         Global Flags:
               --debug            enable debug logging
           -o, --output type      output type: text or json (default text)
           -p, --profile string   ~/.databrickscfg profile
           -t, --target string    bundle target to use (if applicable)
        -      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
        -=== The command is experimental and off by default
        ->>> musterr [CLI] pipelines test --pipeline-id 1234
        -Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
        +Use "databricks pipelines [command] --help" for more information about a command.
         
        +
         === pytest arguments must be passed with --pytest-arg
         >>> musterr [CLI] pipelines test --pipeline-id 1234 -k orders
        -Error: -k is a pytest argument: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'
        +Error: unknown flag: --pipeline-id
         
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
... (600 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/immutable: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable634675070/002/output.txt
        @@ -4,50 +4,112 @@
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Local test paths are not supported with workspace.sync_mode immutable
         >>> musterr [CLI] pipelines test my_pipeline tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown command "test" for "databricks pipelines"
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Error: cannot resolve test selection "tests/test_orders.py::test_backfill": local test paths are not supported with workspace.sync_mode immutable; pass an absolute workspace path
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
         === Absolute workspace paths and running all tests still work
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        ->>> [CLI] pipelines test my_pipeline --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
        +Exit code: 1

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/pipeline-id: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/pipeline-id/output.txt
        +++ /tmp/TestAcceptpipelinestestpipeline-id519920862/002/output.txt
        @@ -1,50 +1,55 @@
         
         === Relative test selections resolve against the pipeline root_path
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][0] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests --no-wait
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        +>>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests --no-wait
        +Error: unknown flag: --pipeline-id
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/orders/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/other/tests"
        -      ]
        -    }
        -  }
        -}
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -=== Without a root_path, test selections must be workspace paths
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][2] tests
        -Error: cannot resolve test selection "tests": the pipeline has no root_path; use an absolute workspace path such as /Workspace/Users/someone@example.com/tests
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][2] /Workspace/Users/someone@example.com/tests/test_other.py::test_x --no-wait
        -Pipeline: [UUID][2]
        -Update: [UUID][3]
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][2]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_other.py::test_x"
        -      ]
        -    }
        -  }
        -}
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -=== Outside a project, --pipeline-id is required
        ->>> musterr [CLI] pipelines test tests
        -Error: Unable to locate the bundle root: databricks.yml not found.
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        -Run this command from a directory that contains a bundle, or set the DATABRICKS_BUNDLE_ROOT environment variable to the bundle root directory.
        -Alternatively, create a new bundle with 'databricks bundle init'.
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
        +Exit code: 1

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/results/failed: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/results/failed/output.txt
        +++ /tmp/TestAcceptpipelinestestresultsfailed343681223/002/output.txt
        @@ -1,20 +1,53 @@
         
         === Failed, errored and skipped tests
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][0]
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        -Running tests...
        -1 of 4 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED]
        -2 of 4 FAILED tests/test_orders.py::test_backfill .............................. [FAILED in 2.14s]
        -3 of 4 SKIPPED tests/test_orders.py::test_legacy_format ........................ [SKIPPED]
        -4 of 4 ERROR tests/test_orders.py::test_fixture ................................ [ERROR]
        +>>> musterr [CLI] pipelines test --pipeline-id [UUID]
        +Error: unknown flag: --pipeline-id
         
        -Completed with 1 failed and 1 errored tests:
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Failure in tests/test_orders.py::test_backfill
        -  AssertionError: assert 100 == 150
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        -Error in tests/test_orders.py::test_fixture
        -  RuntimeError: fixture orders setup failed
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -Done. PASSED=1 FAILED=1 SKIPPED=1 ERROR=1 TOTAL=4
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/results/incomplete: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/results/incomplete/output.txt
        +++ /tmp/TestAcceptpipelinestestresultsincomplete1399351513/002/output.txt
        @@ -1,11 +1,53 @@
         
         === Compute ends before all results are received
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][0]
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        -Running tests...
        -1 of 2 PASSED tests/test_compute.py::test_small_order .......................... [PASSED]
        +>>> musterr [CLI] pipelines test --pipeline-id [UUID]
        +Error: unknown flag: --pipeline-id
         
        -RUN INCOMPLETE: the update failed before all test results were received.
        -Done. PASSED=1 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=2
        -Error: update [UUID][1] is FAILED: ComputeTerminatedException: Compute terminated unexpectedly.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/results/passed: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/results/passed/output.txt
        +++ /tmp/TestAcceptpipelinestestresultspassed435032578/002/output.txt
        @@ -1,30 +1,55 @@
         
         === All tests pass
        ->>> [CLI] pipelines test --pipeline-id [UUID][0]
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        -Running tests...
        -1 of 2 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED in 1.84s]
        -2 of 2 PASSED tests/test_orders.py::test_backfill .............................. [PASSED in 1.00s]
        +>>> [CLI] pipelines test --pipeline-id [UUID]
        +Error: unknown flag: --pipeline-id
         
        -Done. PASSED=2 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=2
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        ->>> print_requests.py --get //events
        -{
        -  "method": "GET",
        -  "path": "/api/2.0/pipelines/[UUID][0]/events",
        -  "q": {
        -    "filter": "update_id = '[UUID][1]'",
        -    "max_results": "250",
        -    "order_by": "timestamp asc"
        -  }
        -}
        -{
        -  "method": "GET",
        -  "path": "/api/2.0/pipelines/[UUID][0]/events",
        -  "q": {
        -    "filter": "update_id = '[UUID][1]'",
        -    "max_results": "250",
        -    "order_by": "timestamp asc"
        -  }
        -}
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
        +Exit code: 1

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/run-control/canceled: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/run-control/canceled/output.txt
        +++ /tmp/TestAcceptpipelinestestrun-controlcanceled1745177831/002/output.txt
        @@ -1,9 +1,53 @@
         
         === A canceled update is an incomplete run
         >>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1]
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        +Error: unknown flag: --pipeline-id
         
        -RUN INCOMPLETE: the update was canceled before all test results were received.
        -Done. PASSED=0 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=1
        -Error: update [UUID][1] is CANCELED
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/run-control/not-test-update: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/run-control/not-test-update/output.txt
        +++ /tmp/TestAcceptpipelinestestrun-controlnot-test-update274809881/002/output.txt
        @@ -1,4 +1,53 @@
         
         === --resume rejects an update that is not a test update
         >>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1]
        -Error: update [UUID][1] is not a test update; pass the update ID that pipelines test printed
        +Error: unknown flag: --pipeline-id
        +
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/run-control/resume: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/run-control/resume/output.txt
        +++ /tmp/TestAcceptpipelinestestrun-controlresume939597835/002/output.txt
        @@ -1,9 +1,55 @@
         
         === --resume reads the results of an existing update
         >>> [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1]
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        -Running tests...
        -1 of 1 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED in 1.00s]
        +Error: unknown flag: --pipeline-id
         
        -Done. PASSED=1 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=1
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
        +Exit code: 1

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/run-control/timeout: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/run-control/timeout/output.txt
        +++ /tmp/TestAcceptpipelinestestrun-controltimeout3691292612/002/output.txt
        @@ -1,7 +1,53 @@
         
         === --timeout stops waiting but not the update, and prints the command that reads the results
         >>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 1s
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        -Running tests...
        -Error: stopped waiting for update [UUID][1] after 1s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --resume [UUID][1]
        +Error: unknown flag: --pipeline-id
        +
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/selection: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/selection/output.txt
        +++ /tmp/TestAcceptpipelinestestselection3332069669/002/output.txt
        @@ -4,80 +4,59 @@
         Created pipelines.my_pipeline
         Files: 3 uploaded, 0 deleted
         Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Run all tests of the only pipeline
         >>> [CLI] pipelines test --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -=== Select tests by relative path and test ID, with pytest arguments in both --pytest-arg forms
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline tests tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/tests/test_other.py --pytest-arg=-k --pytest-arg=orders and not slow --pytest-arg -m --pytest-arg smoke,fast --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests",
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/tests/test_other.py"
        -      ],
        -      "pytest_args": [
        -        "-k",
        -        "orders and not slow",
        -        "-m",
        -        "smoke,fast"
        -      ]
        -    }
        -  }
        -}
        +Use "databricks pipelines [command] --help" for more information about a command.
         
        -=== Relative paths resolve from the working directory
        ->>> [CLI] pipelines test my_pipeline test_orders.py::test_valid_order --no-wait
        -Recommendation: This command runs the last deployed version of the code
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][3]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_valid_order"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Test selections require the pipeline KEY first
        ->>> musterr [CLI] pipelines test tests/test_orders.py
        -Error: "tests/test_orders.py" is not a pipeline in this bundle (pipelines: my_pipeline). Usage: databricks pipelines test my_pipeline tests/test_orders.py
        +Exit code: 1

latest (v1.25.0): same output as main (688c228)

TestAccept/pipelines/test/submit-error: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: Diff:
        --- pipelines/test/submit-error/output.txt
        +++ /tmp/TestAcceptpipelinestestsubmit-error3541450367/002/output.txt
        @@ -1,13 +1,58 @@
         
         === A submit error from the service is shown as is, and no update ID is printed
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][0]
        -Error: An active update '[UUID][1]' already exists for pipeline '[UUID][2]'.
        +>>> musterr [CLI] pipelines test --pipeline-id [UUID]
        +Error: unknown flag: --pipeline-id
         
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
         >>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +File [OUTPUT_DIR]/out.requests.txt not found
        +
        +Exit code: 1

latest (v1.25.0): same output as main (688c228)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 744d6ca

Run: 42728434028

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 271 24 5:18
✅​ aws-windows-latest-4core-16gb 273 22 4:31
✅​ azure linux-2core-8gb 270 24 6:29
✅​ azure-windows-latest-4core-16gb 272 22 5:40
🔄​ gcp linux-2core-8gb 2 269 24 5:51
✅​ gcp-windows-latest-4core-16gb 273 22 6:34
Test Name gcp linux-2core-8gb
🔄​ TestFsCpFileToFileFileNotOverwritten 🔄​f
🔄​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_dbfs 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:13 gcp-windows-latest-4core-16gb TestAccept
3:12 aws-windows-latest-4core-16gb TestAccept
3:06 azure-windows-latest-4core-16gb TestAccept

@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch 2 times, most recently from cb0a526 to 025649f Compare November 3, 2026 00:53
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr3-junit branch 2 times, most recently from b4df318 to 7c61f68 Compare November 3, 2026 17:34
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from 025649f to 23e0bb0 Compare November 3, 2026 17:34
Comment thread cmd/pipelines/test_junit.go Outdated
}
data = append([]byte(xml.Header), data...)
data = append(data, '\n')
if err := os.WriteFile(path, data, 0o644); err != nil {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.WriteFile truncates first, so a crash mid-write leaves a half-written file that CI reads as broken. Could we use atomicfile.Write from libs/atomicfile? bundle/deploy/state_pull.go already uses it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The report is written with atomicfile.Write.

Comment thread cmd/pipelines/test_junit.go Outdated
// A parameter ID in brackets, such as test_cast[a::int], stays with the test name.
func junitCaseName(testID string) (string, string) {
params := ""
if i := strings.Index(testID, "["); i >= 0 {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strings.Index(testID, "[") searches the whole ID, so tests/[weird]/test_a.py::test_x gets classname "" and name tests.[weird].test_a.py::test_x. Searching only after the first :: fixes it. Worth adding that case to TestJUnitCaseName.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. junitCaseName looks for [ only after the first ::, and TestJUnitCaseName has the tests/[weird]/test_a.py::test_x case.


if problem != "" {
suite.Cases = append(suite.Cases, junitTestCase{
Name: "update " + updateID,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entry is named update <id>, while pre-update errors are named pipelines test (line 167). On the timeout path, the update is still running and the message doesn't start with RUN INCOMPLETE:. Could we use one naming scheme, with a clear message for the local-timeout case?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeout entry now says "stopped waiting for update X after 10m; the update keeps running, to get its results run: …" (#7434). One naming scheme for the entries: I'll do that in a follow-up.

Comment thread cmd/pipelines/test.go Outdated
// Every exit writes the report, so that an older report never survives.
report := &testReport{path: junitPath, out: cmd.OutOrStdout(), suiteName: pipelineID}
if report.suiteName == "" && len(args) > 0 {
report.suiteName = args[0]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outside a project, positional args are selections, so pipelines test tests/test_x.py --junit-xml r.xml names the suite tests/test_x.py, and with no args the suite name is empty. Could we use the pipeline name or ID once it resolves?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #7434. Once the pipeline resolves, the suite name is the KEY, or the pipeline ID with --pipeline-id.

Comment thread cmd/pipelines/test.go Outdated

--junit-xml writes one entry per test with its outcome and duration. An
incomplete run, a run without tests, or a timeout adds a failing entry that
names the cause. With workspace.sync_mode immutable, test names keep the path

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: thanks for documenting this. Classnames that change with each deploy break CI test-history views (flaky detection, trends). A follow-up could name cases relative to the snapshot root.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that in a follow-up.

trace musterr $CLI pipelines test --pipeline-id 1234 --no-wait --junit-xml out.no-wait.xml

title "An invalid flag replaces an older report"
echo '<testsuites><testsuite name="1234" tests="1"><testcase name="test_old"></testcase></testsuite></testsuites>' > out.flag-error.xml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: the "an older report is replaced" check now appears in five scripts: arguments, submit-error, run-control, outside, and an invalid project. Each shows the same thing, that a failure before the update still rewrites the report. One case before the request and one at submit would cover it, and the other goldens could go.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Only arguments (before the request) and submit-error (at submit) check it now.

@tharunbhuk-db
tharunbhuk-db marked this pull request as ready for review November 5, 2026 16:09
@tharunbhuk-db tharunbhuk-db changed the title [WIP] Write a JUnit XML report from databricks pipelines test Write a JUnit XML report from databricks pipelines test Nov 5, 2026
@tharunbhuk-db
tharunbhuk-db marked this pull request as draft November 9, 2026 23:57
@tharunbhuk-db
tharunbhuk-db changed the base branch from tharun/pipelines-test-pr2-output to tharun/pipelines-test-a4-run-control November 10, 2026 00:25
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-a4-run-control branch from cd64420 to 17f12d7 Compare November 10, 2026 00:25
tharunbhuk-db added a commit that referenced this pull request Nov 10, 2026
This commit adds the changes of #7297, #7425, #7298, #7426 and #7299, so that
the next commit can build on both stacks. It is not for review here, and goes
away when that stack merges.
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr3-junit branch from 744d6ca to 58df287 Compare November 10, 2026 04:14
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-a4-run-control branch from 17f12d7 to 2f8b6d9 Compare November 10, 2026 04:14
@tharunbhuk-db tharunbhuk-db changed the title Write a JUnit XML report from databricks pipelines test Build a JUnit XML report from the results of a test run Nov 10, 2026
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-a4-run-control branch from 2f8b6d9 to e225070 Compare November 10, 2026 06:57
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr3-junit branch from 58df287 to 9d83e17 Compare November 10, 2026 07:11
@tharunbhuk-db

Copy link
Copy Markdown
Contributor Author

Follow-ups from review:

  • Use one naming scheme for the JUnit entries that are not tests (update <id> and pipelines test). (thread)
  • Name cases relative to the snapshot root with workspace.sync_mode immutable, so classnames stay the same across deploys. (thread)

@tharunbhuk-db
tharunbhuk-db marked this pull request as ready for review November 10, 2026 08:25
@tharunbhuk-db
tharunbhuk-db requested a review from a team as a code owner November 10, 2026 08:25
@tharunbhuk-db
tharunbhuk-db removed the request for review from a team November 10, 2026 08:25
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-a4-run-control branch from e225070 to 1059c51 Compare November 10, 2026 09:08
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr3-junit branch from 9d83e17 to 062047b Compare November 10, 2026 09:08
The report has one testcase per collected test case, named as pytest's JUnit
report names it, with its outcome and duration. A case without a final outcome
is an error, and a run problem, such as an incomplete run, adds a failing entry.
A bracket in a folder name stays part of the path, and the file is replaced
atomically so that a reader never sees a partly written report. The command
does not write the report yet.

Co-authored-by: Isaac <no-reply@databricks.com>
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-a4-run-control branch from 1059c51 to 66654a5 Compare November 10, 2026 17:14
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr3-junit branch from 062047b to c7628c9 Compare November 10, 2026 17:14

@pranshupand-db pranshupand-db left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Follow-ups for the stack are in the #7297 review: #7297

idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 11, 2026
## 🥞 Stacked PR
Use this [link](https://github.com/databricks/cli/pull/7297/files) to
review incremental changes.
-
[**stack/pipelines-test-pr1-command**](databricks#7297)
[[Files changed](https://github.com/databricks/cli/pull/7297/files)] ←
_this PR_
-
[stack/pipelines-test-bundle-key](databricks#7431)
[[Files changed](https://github.com/databricks/cli/pull/7431/files)]
-
[stack/pipelines-test-read-events](databricks#7432)
[[Files changed](https://github.com/databricks/cli/pull/7432/files)]
-
[stack/pipelines-test-print-results](databricks#7433)
[[Files changed](https://github.com/databricks/cli/pull/7433/files)]
-
[stack/pipelines-test-pr2-output](databricks#7298)
[[Files changed](https://github.com/databricks/cli/pull/7298/files)]
-
[stack/pipelines-test-a4-run-control](databricks#7426)
[[Files changed](https://github.com/databricks/cli/pull/7426/files)]
-
[stack/pipelines-test-pr3-junit](databricks#7299)
[[Files changed](https://github.com/databricks/cli/pull/7299/files)]
-
[stack/pipelines-test-junit-flag](databricks#7434)
[[Files changed](https://github.com/databricks/cli/pull/7434/files)]

---------
- Add `databricks pipelines test --pipeline-id <id>
[TEST_SELECTION...]`: starts a test-only update and prints its IDs.
- Experimental: runs only with `DATABRICKS_CLI_PIPELINES_TEST=true`.
- Selections are workspace paths or paths relative to `root_path`, sent
as typed. `--pytest-arg` accepts only `-k` and `-m`.

Tests: unit tests; acceptance in
`acceptance/pipelines/test/{arguments,pipeline-id,submit-error}`.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 11, 2026
## 🥞 Stacked PR
-
[stack/pipelines-test-pr1-command](databricks#7297)
[[Files changed](https://github.com/databricks/cli/pull/7297/files)]
-
[**stack/pipelines-test-bundle-key**](databricks#7431)
[[Files changed](https://github.com/databricks/cli/pull/7431/files)] ←
_this PR_
-
[stack/pipelines-test-read-events](databricks#7432)
[[Files changed](https://github.com/databricks/cli/pull/7432/files)]
-
[stack/pipelines-test-print-results](databricks#7433)
[[Files changed](https://github.com/databricks/cli/pull/7433/files)]
-
[stack/pipelines-test-pr2-output](databricks#7298)
[[Files changed](https://github.com/databricks/cli/pull/7298/files)]
-
[stack/pipelines-test-a4-run-control](databricks#7426)
[[Files changed](https://github.com/databricks/cli/pull/7426/files)]
-
[stack/pipelines-test-pr3-junit](databricks#7299)
[[Files changed](https://github.com/databricks/cli/pull/7299/files)]
-
[stack/pipelines-test-junit-flag](databricks#7434)
[[Files changed](https://github.com/databricks/cli/pull/7434/files)]

---------
- In a bundle, identify the pipeline by KEY: `pipelines test [flags]
[KEY] [TEST_SELECTION...]`.
- Relative selections are local paths, mapped to their deployed paths.
- A first argument that is not a KEY gets an error that names the
bundle's pipelines and the working command.

Tests: unit tests; acceptance in
`acceptance/pipelines/test/{selection,immutable}`.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants