Skip to content

Re-key grants when their parent's ID changes - #7345

Merged
denik merged 8 commits into
mainfrom
denik/grants-rekey
Nov 4, 2026
Merged

denik merged 8 commits into
mainfrom
denik/grants-rekey

Conversation

@denik

@denik denik commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Grants are keyed by their parent's full name (volume/main.s.vol). When the parent got a new ID, the grants were applied under the new name, but their state kept the old ID. This happens on a rename (catalogs, volumes) or a recreate under a new name (schemas). The next bundle plan then proposed the grants again. On UC the old name 404s, so the plan shows create ...grants once; on the test server the plan shows update ...grants forever.

full_name is now an updatable_id_fields entry for every grants type, and DoUpdateWithID records the new ID. This follows secret_scopes.permissions.yml, which solves the same problem for secret scope ACLs. The rule lives in a single configs/grants.yml: a config file named after a sub-resource now applies to that sub-resource of every parent (catalogs.grants, schemas.grants, ...).

Needed before #7343 (schema rename), so that renamed schemas keep their grants in sync.

Tests

New acceptance tests, Cloud = true, passing locally and on AWS:

  • grants/volume_change_name: volume rename (update_id).
  • grants/schemas/change_name: schema name change (a recreate today).

The first commit adds them with the pre-fix goldens; the second commit shows the fix flipping them to a no-op plan.

This pull request and its description were written by Isaac.

denik and others added 2 commits November 4, 2026 13:55
Co-authored-by: Isaac <no-reply@databricks.com>
The grants ID embeds the parent's full name. When the parent was renamed
(catalogs, volumes) or recreated under a new name (schemas), the grants were
applied to the new name but their state kept the old ID, so the next plan
proposed them again: UC 404s on the old name (plan: create), the test server
does not (plan: update, forever).

full_name is now an updatable ID field of every grants type, and
DoUpdateWithID records the new ID.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the DABs DABs related issues label Nov 4, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Commit: ecf3adaff | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0
test branch main (ef99113) latest (v1.24.0)
TestAccept/bundle/resources/grants/schemas/change_name/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/grants/schemas/change_name/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/grants/volume_change_name/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/grants/volume_change_name/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/grants/schemas/change_name/DMS=: branch ✅ | main (ef99113) ❌ | latest (v1.24.0) ❌

main (ef99113):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/schemas/change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsschemaschange_nameDMS=3769790890/002/output.txt
        @@ -43,8 +43,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME]",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME]"
        @@ -106,11 +105,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py schemas schema1.grants id
        -schemas schema1.grants id='schema/main.test-schema-renamed-[UNIQUE_NAME]'
        +schemas schema1.grants id='schema/main.test-schema-[UNIQUE_NAME]'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
        +update schemas.schema1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 1 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-renamed-4iliezzjv5hidkvzqoyoz6krki",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }

latest (v1.24.0):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/schemas/change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsschemaschange_nameDMS=192986567/002/output.txt
        @@ -43,8 +43,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME]",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME]"
        @@ -106,11 +105,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py schemas schema1.grants id
        -schemas schema1.grants id='schema/main.test-schema-renamed-[UNIQUE_NAME]'
        +schemas schema1.grants id='schema/main.test-schema-[UNIQUE_NAME]'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
        +update schemas.schema1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 1 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-renamed-fcxc7nnd2rdv5f5kvb76toot6a",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }
TestAccept/bundle/resources/grants/schemas/change_name/DMS=true: branch ✅ | main (ef99113) ❌ | latest (v1.24.0) ❌

main (ef99113):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/schemas/change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsschemaschange_nameDMS=true372179663/002/output.txt
        @@ -43,8 +43,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME]",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME]"
        @@ -106,11 +105,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py schemas schema1.grants id
        -schemas schema1.grants id='schema/main.test-schema-renamed-[UNIQUE_NAME]'
        +schemas schema1.grants id='schema/main.test-schema-[UNIQUE_NAME]'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
        +update schemas.schema1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 1 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-renamed-os4n33pwive4rp3pkuzb2qla4m",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }

latest (v1.24.0):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/schemas/change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsschemaschange_nameDMS=true3778275564/002/output.txt
        @@ -43,8 +43,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME]",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME]"
        @@ -106,11 +105,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py schemas schema1.grants id
        -schemas schema1.grants id='schema/main.test-schema-renamed-[UNIQUE_NAME]'
        +schemas schema1.grants id='schema/main.test-schema-[UNIQUE_NAME]'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
        +update schemas.schema1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 1 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-renamed-qmgfnsntkjh7dnsmyk5ci7gpim",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }
FAIL
TestAccept/bundle/resources/grants/volume_change_name/DMS=: branch ✅ | main (ef99113) ❌ | latest (v1.24.0) ❌

main (ef99113):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/volume_change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsvolume_change_nameDMS=814414104/002/output.txt
        @@ -54,8 +54,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME].myvolume",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME].myvolume"
        @@ -107,11 +106,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py volumes volume1.grants id
        -volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume_renamed'
        +volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
        +update volumes.volume1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 2 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/volumes/main.test-schema-4rgklwvscvamhnzt5ghsemcdwu.myvolume_renamed"
        LOG.destroy_requests: }
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-4rgklwvscvamhnzt5ghsemcdwu",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }

latest (v1.24.0):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/volume_change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsvolume_change_nameDMS=65687098/002/output.txt
        @@ -54,8 +54,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME].myvolume",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME].myvolume"
        @@ -107,11 +106,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py volumes volume1.grants id
        -volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume_renamed'
        +volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
        +update volumes.volume1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 2 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/volumes/main.test-schema-fgj24qffkjdtdatgkkgymc4ucu.myvolume_renamed"
        LOG.destroy_requests: }
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-fgj24qffkjdtdatgkkgymc4ucu",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }
TestAccept/bundle/resources/grants/volume_change_name/DMS=true: branch ✅ | main (ef99113) ❌ | latest (v1.24.0) ❌

main (ef99113):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/volume_change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsvolume_change_nameDMS=true2133466883/002/output.txt
        @@ -54,8 +54,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME].myvolume",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME].myvolume"
        @@ -107,11 +106,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py volumes volume1.grants id
        -volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume_renamed'
        +volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
        +update volumes.volume1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 2 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/volumes/main.test-schema-w26f3abph5fgjav5aw3v5il5u4.myvolume_renamed"
        LOG.destroy_requests: }
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-w26f3abph5fgjav5aw3v5il5u4",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }
FAIL

latest (v1.24.0):

acceptance_test.go:1342: Diff:
        --- bundle/resources/grants/volume_change_name/output.txt
        +++ /tmp/TestAcceptbundleresourcesgrantsvolume_change_nameDMS=true2188130323/002/output.txt
        @@ -54,8 +54,7 @@
         >>> [CLI] bundle plan --output json
         {
           "full_name": {
        -    "action": "update_id",
        -    "reason": "id_changes",
        +    "action": "update",
             "old": "main.test-schema-[UNIQUE_NAME].myvolume",
             "new": "",
             "remote": "main.test-schema-[UNIQUE_NAME].myvolume"
        @@ -107,11 +106,13 @@
         
         === The grants are recorded under the new name, so the next plan is a no-op
         >>> read_state.py volumes volume1.grants id
        -volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume_renamed'
        +volumes volume1.grants id='volume/main.test-schema-[UNIQUE_NAME].myvolume'
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
        +update volumes.volume1.grants
         
        +Plan: 0 to add, 1 to change, 0 to delete, 2 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.schemas.schema1
        
    acceptance_test.go:1120: 
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/volumes/main.test-schema-znhjk3vf5bci5ormmqtvxasmbi.myvolume_renamed"
        LOG.destroy_requests: }
        LOG.destroy_requests: {
        LOG.destroy_requests:   "method": "DELETE",
        LOG.destroy_requests:   "path": "/api/2.1/unity-catalog/schemas/main.test-schema-znhjk3vf5bci5ormmqtvxasmbi",
        LOG.destroy_requests:   "q": {
        LOG.destroy_requests:     "force": "true"
        LOG.destroy_requests:   }
        LOG.destroy_requests: }

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: ecf3ada

Run: 41853032938

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 277 15 5:38
✅​ aws-windows-latest-4core-16gb 279 13 4:39
✅​ azure linux-2core-8gb 276 15 5:41
✅​ azure-windows-latest-4core-16gb 278 13 4:29
✅​ gcp linux-2core-8gb 277 15 6:17
✅​ gcp-windows-latest-4core-16gb 279 13 4:42
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:33 gcp-windows-latest-4core-16gb TestAccept
3:28 azure-windows-latest-4core-16gb TestAccept
3:16 aws-windows-latest-4core-16gb TestAccept

A config file named after a sub-resource now applies to that sub-resource of
every parent, so the ten identical <parent>.grants.yml files become one.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik marked this pull request as ready for review November 4, 2026 14:43
@denik
denik requested review from a team as code owners November 4, 2026 14:43
Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik enabled stacked auto-merge November 4, 2026 14:46
denik and others added 3 commits November 4, 2026 15:49
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit e1b8c92 Nov 4, 2026
28 checks passed
@denik
denik deleted the denik/grants-rekey branch November 4, 2026 15:33
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: e1b8c92

Run: 41859137575

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
🔄​ aws linux-2core-8gb 8 1509 1255 312:21
❌​ aws-windows-latest-4core-16gb 21 1415 1282 254:02
✅​ azure linux-2core-8gb 1364 1307 136:45
✅​ azure-windows-latest-4core-16gb 1283 1334 141:48
🔄​ gcp linux-2core-8gb 2 1349 1312 156:01
🔄​ gcp-windows-latest-4core-16gb 2 1268 1339 161:49
26 interesting tests: 21 FAIL, 5 flaky
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
❌​ TestAccept ✅​p ❌​F ✅​p ✅​p
🔄​ TestAccept/bundle/debug/fetch-repository-info/DMS= ✅​p ✅​p 🔄​f 🔄​f
🔄​ TestAccept/bundle/debug/fetch-repository-info/DMS=true ✅​p ✅​p 🔄​f 🔄​f
❌​ TestAccept/bundle/deploy/spark-jar-task ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/deploy/spark-jar-task/DMS= 🔄​f ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/deploy/spark-jar-task/DMS=true 🔄​f ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/delete_idempotent ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN= ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1 ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1 ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS= ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/lifecycle-started ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/lifecycle-started-toggle ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS= 🔄​f ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true 🔄​f ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/resize-terminated-fallback 🔄​f ❌​F ✅​p ✅​p
🔄​ TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS= 🔄​f ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true ✅​p ❌​F ✅​p ✅​p
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate 🔄​f ✅​p 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=true 🔄​f ✅​p
Top 50 slowest tests (at least 2 minutes):
duration env testname
30:26 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
23:01 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=
22:54 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
18:44 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
17:22 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
16:29 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
14:51 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
14:50 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
13:35 aws linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
10:56 aws linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=
9:28 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:12 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:08 azure linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
9:02 aws linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=
8:58 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:49 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:44 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:41 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:35 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=true
8:25 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:19 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
8:13 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:08 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
8:08 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
7:44 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:41 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:33 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:32 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
7:19 aws linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:17 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:13 aws linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=true
7:09 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:06 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=true
7:06 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:03 aws linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=
6:53 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:50 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:40 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:40 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=
6:39 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:33 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:21 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:17 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
6:08 aws linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=true
5:49 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
5:48 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
5:45 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
5:42 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=
5:38 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:36 aws linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=

deco-sdk-tagging Bot added a commit that referenced this pull request Nov 9, 2026
## Release v1.25.0

### CLI

 * `databricks aitools install --output json` now reports a more specific `error_category` (`MANIFEST_NOT_FOUND`, `PERMISSION_DENIED`, `FILESYSTEM`) for some failures it reported as `UNCATEGORIZED`. ([#7380](#7380))
 * Suggest `databricks auth logout --profile <profile-name> --delete` when multiple profiles match the same workspace in `databricks auth docker` commands. ([#7363](#7363))
 * Fix `%pip` in Jupyter notebooks opened over `databricks ssh connect` on serverless environment versions 5 and 6 and on dedicated clusters running recent Databricks Runtime versions, which fell back to the stock IPython magic instead of the Databricks one. ([#7263](#7263))

### AI Runtime

 * `databricks air run` now validates the config server-side while it uploads the run artifacts, instead of before, which shortens submission. ([#7368](#7368))
 * Remove `CRITICAL` as a valid `compute.priority_class` value; the AI Runtime API no longer accepts it. ([#7371](#7371))

### Bundles

 * Built-in templates add the workspace ID to `workspace.host` (`?w=<id>`) when the profile or an environment variable sets it, so generated bundles select the right workspace on hosts shared by several workspaces. Welcome messages show the same URL, and `dbt-sql` also adds `?o=<id>` to the HTTP path it suggests for `dbt init`. ([#7350](#7350))
 * Fix grants of a renamed or recreated catalog, schema, volume or other UC resource staying recorded under the old name, which made the next `bundle plan` propose to apply them again. ([#7345](#7345))
 * Fix permissions of a recreated pipeline, app, dashboard, model serving endpoint or other resource staying recorded under the old ID, which made every later `bundle plan` propose to apply them again. ([#7361](#7361))
 * Add the `postgres_data_apis` resource (Private Preview) for enabling and configuring the Data API of a Lakebase Postgres database. ([#7160](#7160))
 * Fixes [#4012](#4012): registered model `aliases` are now set in Unity Catalog on deploy instead of being silently ignored. A model without an `aliases` section keeps the aliases set outside the bundle (for example by MLflow), and `aliases: []` removes them. ([#7163](#7163))
 * Include variable declarations and the source of each resolved value in the `--include-locations` output of `bundle validate` and `bundle summary`. ([#7351](#7351))
 * Add `.workspace.URLID` to bundle templates: the workspace ID that `.workspace.URL` adds, or empty, so templates can use it without failing when the ID is unknown. ([#7388](#7388))
 * Expose the workspace of the selected profile to bundle templates as `.workspace`, with `Hostname`, `ID`, and `URL`, so templates can address a workspace on a host shared by several workspaces. A template input named `workspace` takes precedence. ([#7349](#7349))

### Dependency Updates

 * Bump dependencies with known vulnerabilities. ([#7394](#7394))
 * Upgrade `github.com/mattn/go-runewidth` to v0.0.31, which removes ~30ms of startup time from every command. ([#7391](#7391))
 * Update the Go SDK to [v0.194.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.194.0). ([#7371](#7371))
 * Update the Go SDK to [v0.195.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.195.0). ([#7384](#7384))
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 11, 2026
## Changes
Permissions are keyed by their parent's object ID (`/pipelines/<id>`).
When the parent was recreated under a new ID (a pipeline `storage`
change, an app or model rename, a dashboard `parent_path` change, a
serving endpoint replaced out of band, ...), the permissions were
applied to the new object, but their state kept the old ID. The
permissions API still answers for the deleted object, so every later
`bundle plan` proposed the permissions again and never converged
(verified on AWS).

Same fix as grants in databricks#7345:
`object_id` is an `updatable_id_fields` entry in a shared
`configs/permissions.yml`, and `DoUpdateWithID` records the new ID.
Secret scope ACLs have their own state and keep
`secret_scopes.permissions.yml`; a resource type's own file now replaces
the shared one instead of conflicting with it.

## Tests
- New `permissions/pipelines/recreate` (`Cloud = true`, passing locally
and on AWS): the first commit records the stale ID and the endless plan,
the fix commit flips it to a no-op.
- `model_serving_endpoints/drift/recreated_same_name` had this bug
recorded as `Badness`; it now converges. Passes on AWS.
- `model_serving_endpoints/basic`: the permissions entry is planned as
`update_id` instead of `update`.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 14, 2026
… or schema (databricks#7343)

## Changes
Changing a schema's `name` now renames it in place (`PATCH new_name`)
instead of deleting and recreating it, so the schema and its contents
survive.

UC moves the children of a renamed catalog or schema along with it. A
volume in a renamed schema, or a schema in a renamed catalog
(`schema_name`/`catalog_name: ${resources.<parent>.name}`), used to be
planned as a recreate, whose create then failed with
`RESOURCE_ALREADY_EXISTS` on every deploy. The planner now re-keys such
children (`update_id`, reason `parent_renamed`). Catalog rename was
already supported and had the same latent bug for child schemas.
- A child whose only change is following its parent is `state_only`: the
framework saves it under the ID the adapter's new optional `IDFromState`
gives it, with no request. It is left out of the text plan and counted
as unchanged. A child with its own changes too goes through
`DoUpdateWithID` at its new path.
- A child recreated in the same deploy as its parent's rename is deleted
where the rename moved it: the plan points the delete at the re-keyed ID
(the entry's `id`) and orders it after the parent, taken from the
state's `depends_on` because the config may now reference another one.
Before, the delete went to the old path and the moved volume was left
behind.
- Registered models and UC secrets cannot follow a rename until
databricks#7379 and
databricks#7381, so in a renamed schema they
are recreated where the rename moved them, as on main.

A resource can also be missing from its recorded ID with no rename in
the plan: a deploy stopped after renaming its parent, or it was renamed
out of band. The plan then reads it at the `IDFromState` ID and takes
what it finds only if its internal ID matches the recorded one, so a
different object under that name is never adopted; the plan entry's `id`
(declared, but never set until now) records where it is, and apply
addresses it there. If that read fails with anything but not-found, the
plan warns and goes on as before.
- The new `internal_id_field` lifecycle setting names the internal ID
(`schema_id`, `volume_id`). The framework records it from the remote
state on every save, and the plan skips it (reason `internal_id`).
- A state recorded without it (by an older CLI, or by `bind`) gets it
from a state-only update. Like a skip, it needs no confirmation in
`bind`, and references to it read the remote state. A migration from
terraform records it directly: `schema_id` from the terraform state,
`volume_id` from the remote (read by `bundle deployment migrate`, taken
from the plan check when migrating on deploy).
- With the new state fields, `apitypes.generated.yml` maps schemas and
volumes to `SchemaInfo`/`VolumeInfo`, as registered models already map
to their Info type.

The test server now looks a volume's full name up case-insensitively and
rejects a taken schema name, like UC. `state_present` and the
synced-table terraform fixture no longer create a schema twice. The
earlier test-server work is in
databricks#7408 (merged).

Known gaps, all as on main, recorded with `Badness`: destroy does not
look for a resource renamed out of band
(`schemas/destroy_after_rename_out_of_band`); the plan takes whatever is
at the recorded name without checking the internal ID
(`schemas/foreign_at_recorded_name`); a catalog renamed out of band is
not found (`catalogs/rename_out_of_band`, fixed by
databricks#7418). A parent named by a
literal rather than a reference is not followed, nor is one interpolated
into another field (a quality monitor's `table_name`).

With databricks#7345 and
databricks#7361 (merged), a renamed schema
keeps its grants and permissions in sync. Registered models and UC
secrets follow parent renames in
databricks#7379 and
databricks#7381.

## Tests
Acceptance tests marked `Cloud = true` pass locally and on AWS, GCP and
Azure.
- Renames, `Cloud = true`: `schemas/rename`,
`schemas/rename_with_volume`, `catalogs/rename` (catalog → schema →
volume), `volumes/rename_catalog_and_schema` (two parent renames at
once), `schemas/rename_with_volume_ref` (a job reading the volume's
`full_name`, `id`, `volume_id`), `schemas/rename_then_change` (update,
rename and recreate reach the new path, also after a second rename and a
rename back).
- Rename together with another change (one PATCH):
`schemas/rename_with_comment`, `catalogs/rename_with_comment`. Case-only
renames: `schemas/rename_case_only`, `volumes/rename_case_only` (UC
keeps the lowercase name; a re-key under a renamed schema keeps the
config's casing, which UC reads case-insensitively).
- Recreate during a parent rename: `volumes/move_from_renamed_schema`
(`Cloud = true`), `volumes/recreate_in_renamed_schema` (local only: an
external volume needs an external location),
`schemas/rename_with_model_and_secret` (`Cloud = true`).
- Out of band: `volumes/rename_out_of_band`,
`schemas/rename_out_of_band` (found by id, no requests),
`catalogs/rename_out_of_band`, `catalogs/rename_all_out_of_band`,
`volumes/recreate_out_of_band`, `schemas/recreate_out_of_band` (a
different id: created, not adopted),
`schemas/rename_out_of_band_old_state` (a state without the ids records
them, then follows the rename), `volumes/old_state_ref` (a job reading
the volume's fields stays unchanged while the ids are recorded).
`deployment/bind/volume` now binds without `--auto-approve`.
- Interrupted deploys (local only, kill hook):
`schemas/rename_interrupted`, `schemas/recreate_interrupted`. A failed
lookup read: `schemas/rename_probe_forbidden` (local only, injected
403).
- The key rename tests also deploy from a saved plan
(`EnvMatrix.READPLAN`).
- Migration: `migrate/command/{basic,grants}` record the ids; the
invariant `migrate` and `auto-migrate` targets show no drift.
`grants/schemas/change_name` and `migrate/auto/schema-rename` show the
in-place rename; `lifecycle/prevent-destroy` switched its recreate
trigger from `name` to `catalog_name`.
- Unit tests for `addParentMoves`, `followsParentRename`,
`Changes.HasOnlySingleReason` and the skipped internal-ID changes.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Jan N Rose <janniklas.rose@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants