Repository navigation
Record deployment history for bundle resource bindings #7400
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shreyas-goenka
wants to merge
28
commits into
main
Choose a base branch
from
shreyas-goenka_data/shreyas-goenka/record-bind-history
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
35aa1eb
Record deployment history for bundle resource bindings
shreyas-goenka 738c903
Move shared deployment lookup into DMS library
shreyas-goenka fa0e730
Address bind history self-review findings
shreyas-goenka 4aa7ab6
Keep binding registration separate from existing deploy code
shreyas-goenka 1b5eba7
Move binding deployment registration into bind phase
shreyas-goenka 68f13fb
Explain the immutable binding regression test
shreyas-goenka 92d89a2
Simplify the immutable binding test comment
shreyas-goenka 664ddc1
Clarify immutable binding test coverage
shreyas-goenka 1d118c2
Explain the observed immutable bind regression
shreyas-goenka 41fc4ad
Clarify the immutable bind regression assertion
shreyas-goenka 5d21333
Use requested wording for immutable bind test comment
shreyas-goenka 12a001c
Limit immutable history test to successful binding
shreyas-goenka c8f7245
Record bind history without rewriting resource state
shreyas-goenka 672a8a2
Parse the last recorded deployment version directly
shreyas-goenka 19b0f50
testserver: accept Feature Store resources in DMS versions
shreyas-goenka 2a7dfb1
Remove feature warning workarounds after fixing the DMS stub
shreyas-goenka 52179c9
Normalize summary deployment stamps with nostamp
shreyas-goenka a838ccc
Remove nostamp inline doctests
shreyas-goenka 79c0e13
Explain summary DMS stamp normalization
shreyas-goenka a43bb71
Remove the DMS test-server regression test
shreyas-goenka cbc5ac7
Number the summary stamp normalization comment
shreyas-goenka 1fcd0f0
Publish bind history directly through the DMS SDK
shreyas-goenka 8345600
Remove redundant version parsing unit coverage
shreyas-goenka 8ed612b
Pass binding state through the bind result for history
shreyas-goenka 2c9eee1
Make bind history metadata optional and DMS-specific
shreyas-goenka e2370e1
Inline bind history metadata selection
shreyas-goenka 4139584
Fix moved parser test and stabilize cloud bind preview
shreyas-goenka 7b569b3
Record binds as updates through the shared DMS buffer
shreyas-goenka File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * Record deployment history when binding existing resources with `bundle deployment bind` when deployment history is enabled. ([#7400](https://github.com/databricks/cli/pull/7400)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,7 +12,7 @@ rm -f "$OUT_REQUESTS" | |
| trace $CLI bundle deployment bind foo $NEW_PIPELINE_ID --auto-approve &> out.bind-success.txt | ||
| trace print_requests.py '^//import-file/' '^//workspace/' | ||
|
|
||
| trace $CLI bundle summary -o json | jq .resources > out.summary.json | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added because now the summary contains deployment and version ID |
||
| trace $CLI bundle summary -o json | jq .resources | nostamp.py > out.summary.json | ||
| trace $CLI bundle plan | ||
|
|
||
| trace musterr $CLI bundle deploy | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,7 +9,7 @@ rm -f "$OUT_REQUESTS" | |
| trace $CLI bundle deployment bind foo $NEW_PIPELINE_ID --auto-approve &> out.bind-success.txt | ||
| print_requests.py '^//import-file/' '^//workspace/delete' | ||
|
|
||
| trace $CLI bundle summary -o json > out.summary.json | ||
| trace $CLI bundle summary -o json | nostamp.py > out.summary.json | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated nostamp.py to scrub the deployment ID from the summary output as well. |
||
| trace $CLI bundle plan | ||
|
|
||
| rm -f "$OUT_REQUESTS" | ||
|
|
||
13 changes: 13 additions & 0 deletions
13
acceptance/bundle/dms/bind-history-failures/databricks.yml.tmpl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| bundle: | ||
| name: dms-bind-history-failures-$UNIQUE_NAME | ||
|
|
||
| resources: | ||
| jobs: | ||
| create_failed: | ||
| name: create_failed | ||
| record_failed: | ||
| name: record_failed | ||
| complete_failed: | ||
| name: complete_failed | ||
| upload_failed: | ||
| name: upload_failed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Cloud = false | ||
| EnvMatrix.DMS = ["true"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
|
|
||
| === A version creation failure does not prevent binding | ||
|
|
||
| >>> fault.py POST /api/2.0/bundle/deployments/*/versions 400 0 1 | ||
|
|
||
| >>> [CLI] bundle deployment bind create_failed [JOB_ID] --auto-approve | ||
| Warn: Failed to record bind history: failed to create deployment version: Fault injected by test. | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| === An operation recording failure does not prevent binding | ||
|
|
||
| >>> fault.py PATCH /api/2.0/bundle/** 400 0 1 | ||
|
|
||
| >>> [CLI] bundle deployment bind record_failed [JOB_ID] --auto-approve | ||
| Warn: Failed to record bind history: recording operation for resources.jobs.record_failed: Fault injected by test. | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| === A version completion failure does not prevent binding | ||
|
|
||
| >>> fault.py POST /api/2.0/bundle/deployments/*/versions/*/complete 400 0 1 | ||
|
|
||
| >>> [CLI] bundle deployment bind complete_failed [JOB_ID] --auto-approve | ||
| Warn: Failed to complete deployment history version: Fault injected by test. | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| === All bindings survived the remote state round trips | ||
|
|
||
| >>> jq .state | map_values(.__id__) .databricks/bundle/default/resources.json | ||
| { | ||
| "resources.jobs.complete_failed": "[JOB_ID]", | ||
| "resources.jobs.create_failed": "[JOB_ID]", | ||
| "resources.jobs.record_failed": "[JOB_ID]" | ||
| } | ||
|
|
||
| >>> [CLI] jobs get [JOB_ID] | ||
| { | ||
| "job_id": [JOB_ID], | ||
| "name": "bind-target-[UNIQUE_NAME]" | ||
| } | ||
|
|
||
| >>> print_requests.py --dms //versions --oneline | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "3"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.create_failed"}], "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "5"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.record_failed"}], "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/5/operations/jobs.record_failed", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[JOB_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-bind-history-failures-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"record_failed\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/5/complete", "body": {"completion_reason": "VERSION_COMPLETE_FAILURE"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "7"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.complete_failed"}], "previous_version_id": "5", "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/7/operations/jobs.complete_failed", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[JOB_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-bind-history-failures-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"complete_failed\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/7/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}} | ||
|
|
||
| === A remote state upload failure fails the bind version | ||
|
|
||
| >>> fault.py POST /api/2.0/workspace-files/import-file/**resources.json 400 0 1 | ||
|
|
||
| >>> musterr [CLI] bundle deployment bind upload_failed [JOB_ID] --auto-approve | ||
| Error: Fault injected by test. (400 INJECTED) | ||
|
|
||
| Endpoint: POST [DATABRICKS_URL]/api/2.0/workspace-files/import-file/Workspace%2FUsers%2F[USERNAME]%2F.bundle%2Fdms-bind-history-failures-[UNIQUE_NAME]%2Fdefault%2Fstate%2Fresources.json?overwrite=true | ||
| HTTP Status: 400 Bad Request | ||
| API error_code: INJECTED | ||
| API message: Fault injected by test. | ||
|
|
||
|
|
||
| >>> print_requests.py --dms //versions --oneline | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "9"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.upload_failed"}], "previous_version_id": "7", "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/9/operations/jobs.upload_failed", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[JOB_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-bind-history-failures-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"upload_failed\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/9/complete", "body": {"completion_reason": "VERSION_COMPLETE_FAILURE"}} | ||
|
|
||
| === Remote state still contains only the successful bindings | ||
|
|
||
| >>> [CLI] bundle summary -o json | ||
| { | ||
| "complete_failed": "[JOB_ID]", | ||
| "create_failed": "[JOB_ID]", | ||
| "record_failed": "[JOB_ID]", | ||
| "upload_failed": null | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| envsubst < databricks.yml.tmpl > databricks.yml | ||
| job_id=$($CLI jobs create --json '{"name": "bind-target-'$UNIQUE_NAME'"}' | jq -r '.job_id') | ||
| add_repl "$job_id" JOB_ID | ||
|
|
||
| title "A version creation failure does not prevent binding" | ||
| trace fault.py "POST /api/2.0/bundle/deployments/*/versions" 400 0 1 | ||
| trace $CLI bundle deployment bind create_failed "$job_id" --auto-approve | ||
| rm -rf .databricks | ||
|
|
||
| title "An operation recording failure does not prevent binding" | ||
| trace fault.py "PATCH /api/2.0/bundle/**" 400 0 1 | ||
| trace $CLI bundle deployment bind record_failed "$job_id" --auto-approve | ||
| rm -rf .databricks | ||
|
|
||
| title "A version completion failure does not prevent binding" | ||
| trace fault.py "POST /api/2.0/bundle/deployments/*/versions/*/complete" 400 0 1 | ||
| trace $CLI bundle deployment bind complete_failed "$job_id" --auto-approve | ||
|
|
||
| title "All bindings survived the remote state round trips" | ||
| trace jq '.state | map_values(.__id__)' .databricks/bundle/default/resources.json | ||
| trace $CLI jobs get "$job_id" | jq '{job_id, name: .settings.name}' | ||
| trace print_requests.py --dms //versions --oneline | ||
|
|
||
|
|
||
| title "A remote state upload failure fails the bind version" | ||
| trace fault.py "POST /api/2.0/workspace-files/import-file/**resources.json" 400 0 1 | ||
| trace musterr $CLI bundle deployment bind upload_failed "$job_id" --auto-approve | ||
| trace print_requests.py --dms //versions --oneline | ||
|
|
||
| rm -rf .databricks | ||
| title "Remote state still contains only the successful bindings" | ||
| trace $CLI bundle summary -o json | jq '.resources.jobs | map_values(.id // null)' | ||
| rm -f "$OUT_REQUESTS" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false # Inject failures into the deployment history service. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| bundle: | ||
| name: dms-bind-immutable-$UNIQUE_NAME | ||
| workspace: | ||
| sync_mode: immutable | ||
| resources: | ||
| jobs: | ||
| foo: | ||
| name: foo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Cloud = false | ||
| EnvMatrix.DMS = ["true"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
|
|
||
| >>> [CLI] bundle deployment bind foo [JOB_ID] --auto-approve | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json | ||
| [ | ||
| { | ||
| "last_action_type": "OPERATION_ACTION_TYPE_UPDATE", | ||
| "resource_id": "[JOB_ID]", | ||
| "resource_key": "jobs.foo" | ||
| } | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| envsubst < databricks.yml.tmpl > databricks.yml | ||
| job_id=$($CLI jobs create --json '{"name": "bind-target-'$UNIQUE_NAME'"}' | jq -r '.job_id') | ||
| add_repl "$job_id" JOB_ID | ||
|
|
||
| trace $CLI bundle deployment bind foo "$job_id" --auto-approve | ||
| deployment_id=$(MSYS_NO_PATHCONV=1 $CLI workspace get-status "/Workspace/Users/${CURRENT_USER_NAME}/.bundle/dms-bind-immutable-${UNIQUE_NAME}/default/state/resources.deployment.json" -o json | jq -r '.object_id') | ||
| trace MSYS_NO_PATHCONV=1 $CLI bundle-deployments list-resources "deployments/${deployment_id}" -o json | jq -S 'map({resource_key, resource_id, last_action_type})' | ||
| rm -f "$OUT_REQUESTS" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # This test ensures that deployment history works with bind and immutable folders. This case is interesting | ||
| # because by default we record the snapshot path in deploy, but that is not available during bind. So it's important | ||
| # to ensure that we don't do that during deployment bind, and recording history continues to work. | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,2 @@ | ||
| Cloud = false | ||
| Cloud = true | ||
| EnvMatrix.DMS = ["true"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,2 @@ | ||
| RecordRequests = false | ||
| # Verify shadow history markers and resource IDs across bind, deploy, unbind, and rebind. | ||
| Cloud = false | ||
| Cloud = true |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running the DMS=true variants of the three scripts this PR pipes through nostamp (bind/pipelines/update, bind/pipelines/recreate, bind/dashboard/recreation) with the scripts at their base revision passes locally at this HEAD: after a first bind there is no marker, so summary and the state dump never carry
bundle.deploymentIDs. What difference is shape 5 and these three pipes guarding against, a cloud run or a later PR in the stack? If the latter, can they move there?