Skip to content

Record lifecycle.prevent_destroy in state so removed resources stay protected - #7499

Open
andrewnester wants to merge 4 commits into
mainfrom
factory/4872-prevent-destroy-on-removal
Open

andrewnester wants to merge 4 commits into
mainfrom
factory/4872-prevent-destroy-on-removal

Conversation

@andrewnester

Copy link
Copy Markdown
Contributor

Changes

Fixes #4872. lifecycle.prevent_destroy now also protects a resource after it is removed from the configuration: bundle deploy and bundle destroy refuse to delete it and name the escape hatch (add it back with prevent_destroy: false, deploy, then remove it).

  • The flag is recorded in the deployment state on every state write; old state files load unchanged.
  • A flag-only toggle on an otherwise unchanged resource is persisted as a state rewrite, without a backend call.

Why

The check only read the flag from the current configuration, so a removed resource was never checked and was deleted. That mirrored Terraform but defeated the flag's purpose.

Tests

Updated acceptance/bundle/lifecycle/prevent-destroy (removal refused, escape hatch succeeds, both DMS variants), new acceptance/bundle/dms/prevent-destroy, and unit tests for the state round trip, the rewrite and the new check branch.

This PR was written by Claude Code.

This pull request and its description were written by Isaac.

andrewnester and others added 2 commits November 11, 2026 18:05
A resource with lifecycle.prevent_destroy: true was deleted as soon as it
was removed from the configuration: checkForPreventDestroy only read the
flag from the current configuration, so a resource that is only in the
state was never checked, and the state did not record the flag either.

Record the flag with each state entry (resources.json and the DMS
envelope) and carry it in the plan: from the configuration for declared
resources, from the state for delete entries of removed ones. A toggle
of the flag on an otherwise unchanged resource plans as skip with
prevent_destroy_changed set; applying it rewrites the state entry without
a backend call, and with deployment history it is staged and recorded as
an update, since every recorded write must map to a staged operation.
The check then refuses, for both deploy and destroy, to delete a
resource that is only in the state and was last deployed with the flag.
To delete it, add it back with prevent_destroy: false, deploy, and
remove it again. Resources still in the configuration are governed by
the configured value, as before.

Fixes #4872

Co-authored-by: Isaac <no-reply@databricks.com>
@andrewnester andrewnester added the ai-auto PRs made by AI loops & factories, label Nov 11, 2026
@github-actions github-actions Bot added the DABs DABs related issues label Nov 11, 2026
@andrewnester
andrewnester marked this pull request as ready for review November 11, 2026 18:09
@andrewnester
andrewnester requested review from a team as code owners November 11, 2026 18:09
@andrewnester
andrewnester requested review from anton-107, denik, janniklasrose and pietern and removed request for anton-107 November 11, 2026 18:09
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Commit: bb7c21cc2 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

3 tests: regression=3 unreleased=0 coverage=0 failing=0
test branch main (eef3b54) latest (v1.26.0)
TestAccept/bundle/dms/prevent-destroy/DMS=true ✅ ❌ ❌
TestAccept/bundle/lifecycle/prevent-destroy/DMS= ✅ ❌ ❌
TestAccept/bundle/lifecycle/prevent-destroy/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/prevent-destroy/DMS=true: branch ✅ | main (eef3b54) ❌ | latest (v1.26.0) ❌

main (eef3b54):

acceptance_test.go:1374: Diff:
        --- bundle/dms/prevent-destroy/output.txt
        +++ /tmp/TestAcceptbundledmsprevent-destroyDMS=true3196446235/002/output.txt
        @@ -18,7 +18,7 @@
             "error_message": "",
             "resource_id": "[FOO_ID]",
             "sequence_id": 0,
        -    "state": "{\"state\":{\"deployment\":{\"deployment_id\":\"[DEPLOYMENT_ID]\",\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-prevent-destroy-[UNIQUE_NAME]/default/state/metadata.json\",\"version_id\":\"1\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"foo\",\"queue\":{\"enabled\":true}},\"prevent_destroy\":true}",
        +    "state": "{\"state\":{\"deployment\":{\"deployment_id\":\"[DEPLOYMENT_ID]\",\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-prevent-destroy-[UNIQUE_NAME]/default/state/metadata.json\",\"version_id\":\"1\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"foo\",\"queue\":{\"enabled\":true}}}",
             "status": "OPERATION_STATUS_SUCCEEDED"
           }
         }
        @@ -32,58 +32,10 @@
         
         >>> musterr [CLI] bundle deploy
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-prevent-destroy-[UNIQUE_NAME]/default/files...
        -Error: resources.jobs.foo was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -
        -Files: 1 uploaded, 1 deleted
        -
        ->>> musterr [CLI] bundle destroy --auto-approve
        -Error: resources.jobs.foo was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -
        -
        -=== Adding it back with prevent_destroy: false leaves the job unchanged, but records the flag as an update of its state
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-prevent-destroy-[UNIQUE_NAME]/default/files...
        -Files: 1 uploaded, 1 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 1 unchanged
        -
        ->>> print_requests.py --dms //versions/2/operations
        -{
        -  "method": "PATCH",
        -  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/jobs.foo",
        -  "q": {
        -    "update_mask": "state,error_message,resource_id,status"
        -  },
        -  "body": {
        -    "error_message": "",
        -    "resource_id": "[FOO_ID]",
        -    "sequence_id": 0,
        -    "state": "{\"state\":{\"deployment\":{\"deployment_id\":\"[DEPLOYMENT_ID]\",\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-prevent-destroy-[UNIQUE_NAME]/default/state/metadata.json\",\"version_id\":\"1\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"foo\",\"queue\":{\"enabled\":true}}}",
        -    "status": "OPERATION_STATUS_SUCCEEDED"
        -  }
        -}
        -
        -=== The job can now be removed
        -
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-prevent-destroy-[UNIQUE_NAME]/default/files...
         Deleted jobs.foo
        -Files: 0 uploaded, 1 deleted
        +Files: 1 uploaded, 1 deleted
         Resources: 0 created, 0 changed, 1 deleted, 0 unchanged
         
        ->>> print_requests.py --dms //versions/3/operations
        -{
        -  "method": "PATCH",
        -  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/operations/jobs.foo",
        -  "q": {
        -    "update_mask": "state,error_message,status"
        -  },
        -  "body": {
        -    "error_message": "",
        -    "sequence_id": 0,
        -    "status": "OPERATION_STATUS_SUCCEEDED"
        -  }
        -}
        +Unexpected success
        +
        +Exit code: 1
        
    acceptance_test.go:1164: Test produced unexpected files:
        removed/job.yml

latest (v1.26.0): same output as main (eef3b54)

TestAccept/bundle/lifecycle/prevent-destroy/DMS=: branch ✅ | main (eef3b54) ❌ | latest (v1.26.0) ❌

main (eef3b54):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate\nName: prevent-destroy\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default\n\nValidation OK!\n\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...\nCreated pipelines.my_pipelines\nCreated schemas.my_schema\nFiles: 4 uploaded, 0 deleted\nResources: 2 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> errcode [CLI] bundle plan\nPlan: 0 to add, 0 to change, 0 to delete, 2 unchanged\n"
        	            	actual  : "\n>>> [CLI] bundle validate\nName: prevent-destroy\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default\n\nValidation OK!\n\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...\nCreated pipelines.my_pipelines\nCreated schemas.my_schema\nFiles: 4 uploaded, 0 deleted\nResources: 2 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> errcode [CLI] bundle plan\nPlan: 0 to add, 0 to change, 0 to delete, 2 unchanged\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -20 +20,3 @@
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/lifecycle/prevent-destroy/DMS=
        	Messages:   	bundle/lifecycle/prevent-destroy/output.txt vs /tmp/TestAcceptbundlelifecycleprevent-destroyDMS=619739195/002/output.txt
    acceptance_test.go:1374: Diff:
        --- bundle/lifecycle/prevent-destroy/out.txt
        +++ /tmp/TestAcceptbundlelifecycleprevent-destroyDMS=619739195/001/out.txt
        @@ -50,13 +50,13 @@
         {
           "resources.pipelines.my_pipelines": {
             "action": "skip",
        -    "prevent_destroy": true,
        -    "prevent_destroy_changed": true
        +    "prevent_destroy": null,
        +    "prevent_destroy_changed": null
           },
           "resources.schemas.my_schema": {
             "action": "skip",
        -    "prevent_destroy": true,
        -    "prevent_destroy_changed": true
        +    "prevent_destroy": null,
        +    "prevent_destroy_changed": null
           }
         }
         
        @@ -73,30 +70,7 @@
         
         >>> musterr [CLI] bundle deploy --auto-approve
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...
        -Error: resources.pipelines.my_pipelines was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -resources.schemas.my_schema was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
         
        -Files: 3 uploaded, 2 deleted
        -
        ->>> musterr [CLI] bundle destroy --auto-approve
        -Error: resources.pipelines.my_pipelines was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -resources.schemas.my_schema was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -
        -
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...
        -Files: 3 uploaded, 2 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
        -
        ->>> errcode [CLI] bundle plan
        -delete pipelines.my_pipelines
        -delete schemas.my_schema
        -
        -Plan: 0 to add, 0 to change, 2 to delete, 0 unchanged
        -
        ->>> [CLI] bundle deploy --auto-approve
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...
        -
         This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
           delete resources.schemas.my_schema
         
        @@ -107,5 +80,7 @@
           delete resources.pipelines.my_pipelines
         Deleted pipelines.my_pipelines
         Deleted schemas.my_schema
        -Files: 1 uploaded, 2 deleted
        +Files: 3 uploaded, 2 deleted
         Resources: 0 created, 0 changed, 2 deleted, 0 unchanged
        +
        +Unexpected success
        
    acceptance_test.go:1164: Test produced unexpected files:
        removed/pipeline.yml
        removed/schema.yml

latest (v1.26.0): same output as main (eef3b54)

TestAccept/bundle/lifecycle/prevent-destroy/DMS=true: branch ✅ | main (eef3b54) ❌ | latest (v1.26.0) ❌

main (eef3b54):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate\nName: prevent-destroy\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default\n\nValidation OK!\n\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...\nCreated pipelines.my_pipelines\nCreated schemas.my_schema\nFiles: 4 uploaded, 0 deleted\nResources: 2 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> errcode [CLI] bundle plan\nPlan: 0 to add, 0 to change, 0 to delete, 2 unchanged\n"
        	            	actual  : "\n>>> [CLI] bundle validate\nName: prevent-destroy\nTarget: default\nWorkspace:\n  User: [USERNAME]\n  Path: /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default\n\nValidation OK!\n\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...\nCreated pipelines.my_pipelines\nCreated schemas.my_schema\nFiles: 4 uploaded, 0 deleted\nResources: 2 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> errcode [CLI] bundle plan\nPlan: 0 to add, 0 to change, 0 to delete, 2 unchanged\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -20 +20,3 @@
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/lifecycle/prevent-destroy/DMS=true
        	Messages:   	bundle/lifecycle/prevent-destroy/output.txt vs /tmp/TestAcceptbundlelifecycleprevent-destroyDMS=true475809718/002/output.txt
    acceptance_test.go:1374: Diff:
        --- bundle/lifecycle/prevent-destroy/out.txt
        +++ /tmp/TestAcceptbundlelifecycleprevent-destroyDMS=true475809718/001/out.txt
        @@ -50,13 +50,13 @@
         {
           "resources.pipelines.my_pipelines": {
             "action": "skip",
        -    "prevent_destroy": true,
        -    "prevent_destroy_changed": true
        +    "prevent_destroy": null,
        +    "prevent_destroy_changed": null
           },
           "resources.schemas.my_schema": {
             "action": "skip",
        -    "prevent_destroy": true,
        -    "prevent_destroy_changed": true
        +    "prevent_destroy": null,
        +    "prevent_destroy_changed": null
           }
         }
         
        @@ -73,30 +70,7 @@
         
         >>> musterr [CLI] bundle deploy --auto-approve
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...
        -Error: resources.pipelines.my_pipelines was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -resources.schemas.my_schema was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
         
        -Files: 3 uploaded, 2 deleted
        -
        ->>> musterr [CLI] bundle destroy --auto-approve
        -Error: resources.pipelines.my_pipelines was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -resources.schemas.my_schema was removed from the configuration, but its last deployment has lifecycle.prevent_destroy set. To delete it, add it back with lifecycle.prevent_destroy: false and deploy, then remove it again
        -
        -
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...
        -Files: 3 uploaded, 2 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
        -
        ->>> errcode [CLI] bundle plan
        -delete pipelines.my_pipelines
        -delete schemas.my_schema
        -
        -Plan: 0 to add, 0 to change, 2 to delete, 0 unchanged
        -
        ->>> [CLI] bundle deploy --auto-approve
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/prevent-destroy/default/files...
        -
         This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
           delete resources.schemas.my_schema
         
        @@ -107,5 +80,7 @@
           delete resources.pipelines.my_pipelines
         Deleted pipelines.my_pipelines
         Deleted schemas.my_schema
        -Files: 1 uploaded, 2 deleted
        +Files: 3 uploaded, 2 deleted
         Resources: 0 created, 0 changed, 2 deleted, 0 unchanged
        +
        +Unexpected success
        
    acceptance_test.go:1164: Test produced unexpected files:
        removed/pipeline.yml
        removed/schema.yml
FAIL

latest (v1.26.0): same output as main (eef3b54)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: bb7c21c

Run: 43414626999

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 16 10:14
💚​ aws-windows-latest-4core-16gb 1 274 14 5:29
💚​ azure linux-2core-8gb 1 271 16 11:19
💚​ azure-windows-latest-4core-16gb 1 273 14 6:09
💚​ gcp linux-2core-8gb 1 272 16 11:25
💚​ gcp-windows-latest-4core-16gb 1 274 14 8:25
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 11 slowest tests (at least 2 minutes):
duration env testname
4:19 azure linux-2core-8gb TestAccept
3:55 aws linux-2core-8gb TestAccept
3:53 gcp linux-2core-8gb TestAccept
3:49 gcp-windows-latest-4core-16gb TestAccept
3:20 azure linux-2core-8gb TestAccept/bundle/dms/prevent-destroy/DMS=true
3:09 azure-windows-latest-4core-16gb TestAccept
3:08 gcp-windows-latest-4core-16gb TestFilerReadWrite/workspace_files
3:07 aws-windows-latest-4core-16gb TestAccept
3:01 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:34 gcp linux-2core-8gb TestAccept/bundle/dms/prevent-destroy/DMS=true
2:19 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/prevent-destroy/DMS=true

@andrewnester andrewnester added Enhancement New feature or request and removed ai-auto PRs made by AI loops & factories, labels Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues Enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lifecycle.prevent_destroy flag is ignored in bundle operations

2 participants