Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions acceptance/pipelines/databricks-cli-help/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Main Commands
run Run a pipeline
show Preview a table's columns and sample rows
stop Stop a pipeline
test *Preview* Run the tests of a pipeline

Management Commands
apply-environment *Public Preview* Apply the latest environment to the pipeline.
Expand Down
1 change: 1 addition & 0 deletions acceptance/pipelines/test/arguments/out.test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Cloud = false
61 changes: 61 additions & 0 deletions acceptance/pipelines/test/arguments/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@

>>> [CLI] pipelines test --help
Run the tests of a pipeline in a test-only update.

This command is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it.

Identify the pipeline with --pipeline-id.

TEST_SELECTION is a test directory, file, or pytest test ID, for example
tests/test_orders.py::test_backfill. A relative path is relative to the
pipeline's root_path. Paths that start with / are workspace paths. Without a
selection, all of the pipeline's tests run.

Each --pytest-arg passes one argument to pytest, in the order given. Only -k
and -m are supported, each followed by its value in the next --pytest-arg, for
example:

databricks pipelines test --pipeline-id <id> --pytest-arg=-k --pytest-arg='orders and not slow'

Usage:
databricks pipelines test [flags] [TEST_SELECTION...]

Pipeline Test Flags:
--pipeline-id string ID of the pipeline to test.
--pytest-arg stringArray Argument to pass to pytest (repeatable). Only -k and -m are supported.

Flags:
-h, --help help for test

Global Flags:
--debug enable debug logging
-o, --output type output type: text or json (default text)
-p, --profile string ~/.databrickscfg profile
-t, --target string bundle target to use (if applicable)
--var strings set values for variables defined in project config. Example: --var="foo=bar"

=== The command is experimental and off by default
>>> musterr [CLI] pipelines test --pipeline-id 1234
Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it

=== pytest arguments must be passed with --pytest-arg
>>> musterr [CLI] pipelines test --pipeline-id 1234 -k orders
Error: -k is a pytest argument: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'

>>> musterr [CLI] pipelines test --pipeline-id 1234 -- -k orders
Error: the -- separator is not supported: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'

=== Invalid --pytest-arg values fail before any request
>>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=-x
Error: unsupported pytest argument "-x": only -k and -m are supported

>>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=
Error: --pytest-arg requires a non-empty value

=== The pipeline ID must be a pipeline ID
>>> musterr [CLI] pipelines test --pipeline-id 1234
Error: --pipeline-id "1234" is not a pipeline ID

=== JSON output is not supported
>>> musterr [CLI] pipelines test --pipeline-id 1234 -o json
Error: --output json is not supported by pipelines test
18 changes: 18 additions & 0 deletions acceptance/pipelines/test/arguments/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
trace $CLI pipelines test --help

title "The command is experimental and off by default"
(unset DATABRICKS_CLI_PIPELINES_TEST; trace musterr $CLI pipelines test --pipeline-id 1234)

title "pytest arguments must be passed with --pytest-arg"
trace musterr $CLI pipelines test --pipeline-id 1234 -k orders
trace musterr $CLI pipelines test --pipeline-id 1234 -- -k orders

title "Invalid --pytest-arg values fail before any request"
trace musterr $CLI pipelines test --pipeline-id 1234 --pytest-arg=-x
trace musterr $CLI pipelines test --pipeline-id 1234 --pytest-arg=""

title "The pipeline ID must be a pipeline ID"
trace musterr $CLI pipelines test --pipeline-id 1234

title "JSON output is not supported"
trace musterr $CLI pipelines test --pipeline-id 1234 -o json
1 change: 1 addition & 0 deletions acceptance/pipelines/test/pipeline-id/out.test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Cloud = false
46 changes: 46 additions & 0 deletions acceptance/pipelines/test/pipeline-id/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@

=== Relative test selections resolve against the pipeline root_path
>>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][0] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests
Pipeline: [UUID][0]
Update: [UUID][1]

>>> print_requests.py //updates
{
"method": "POST",
"path": "/api/2.0/pipelines/[UUID][0]/updates",
"body": {
"test_only": true,
"test_options": {
"test_selection": [
"/Workspace/Users/someone@example.com/orders/tests/test_orders.py::test_backfill",
"/Workspace/Users/someone@example.com/other/tests"
]
}
}
}

=== Without a root_path, test selections must be workspace paths
>>> musterr [CLI] pipelines test --pipeline-id [UUID][2] tests
Error: cannot resolve test selection "tests": the pipeline has no root_path; use an absolute workspace path such as /Workspace/Users/someone@example.com/tests

>>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][2] /Workspace/Users/someone@example.com/tests/test_other.py::test_x
Pipeline: [UUID][2]
Update: [UUID][3]

>>> print_requests.py //updates
{
"method": "POST",
"path": "/api/2.0/pipelines/[UUID][2]/updates",
"body": {
"test_only": true,
"test_options": {
"test_selection": [
"/Workspace/Users/someone@example.com/tests/test_other.py::test_x"
]
}
}
}

=== --pipeline-id is required
>>> musterr [CLI] pipelines test tests
Error: identify the pipeline with --pipeline-id
15 changes: 15 additions & 0 deletions acceptance/pipelines/test/pipeline-id/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
pipeline_id=$($CLI pipelines create --json '{"name": "orders", "root_path": "/Workspace/Users/someone@example.com/orders"}' | jq -r .pipeline_id)
no_root_pipeline_id=$($CLI pipelines create --json '{"name": "no-root"}' | jq -r .pipeline_id)
rm "$OUT_REQUESTS"

title "Relative test selections resolve against the pipeline root_path"
trace MSYS_NO_PATHCONV=1 $CLI pipelines test --pipeline-id $pipeline_id tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests
trace print_requests.py //updates

title "Without a root_path, test selections must be workspace paths"
trace musterr $CLI pipelines test --pipeline-id $no_root_pipeline_id tests
trace MSYS_NO_PATHCONV=1 $CLI pipelines test --pipeline-id $no_root_pipeline_id /Workspace/Users/someone@example.com/tests/test_other.py::test_x
trace print_requests.py //updates

title "--pipeline-id is required"
trace musterr $CLI pipelines test tests
1 change: 1 addition & 0 deletions acceptance/pipelines/test/submit-error/out.test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Cloud = false
13 changes: 13 additions & 0 deletions acceptance/pipelines/test/submit-error/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@

=== A submit error from the service is shown as is, and no update ID is printed
>>> musterr [CLI] pipelines test --pipeline-id [UUID][0]
Error: An active update '[UUID][1]' already exists for pipeline '[UUID][0]'.

>>> print_requests.py //updates
{
"method": "POST",
"path": "/api/2.0/pipelines/[UUID][0]/updates",
"body": {
"test_only": true
}
}
3 changes: 3 additions & 0 deletions acceptance/pipelines/test/submit-error/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
title "A submit error from the service is shown as is, and no update ID is printed"
trace musterr $CLI pipelines test --pipeline-id 11111111-2222-3333-4444-555555555555
trace print_requests.py //updates
9 changes: 9 additions & 0 deletions acceptance/pipelines/test/submit-error/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[[Server]]
Pattern = "POST /api/2.0/pipelines/{pipeline_id}/updates"
Response.Body = '''
{
"error_code": "RESOURCE_CONFLICT",
"message": "An active update '00000000-0000-0000-0000-000000000001' already exists for pipeline '11111111-2222-3333-4444-555555555555'."
}
'''
Response.StatusCode = 409
4 changes: 4 additions & 0 deletions acceptance/pipelines/test/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
RecordRequests = true
Ignore = ['.databricks']
# The command is experimental; the gate case in arguments turns it off.
Env.DATABRICKS_CLI_PIPELINES_TEST = "true"
1 change: 1 addition & 0 deletions cmd/pipelines/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ func Commands() []*cobra.Command {
destroyCommand(),
runCommand(),
dryRunCommand(),
testCommand(),
historyCommand(),
logsCommand(),
openCommand(),
Expand Down
Loading
Loading