Repository navigation
Read and adopt existing grants on the first bundle plan - #7422
Open
janniklasrose wants to merge 63 commits into
Open
janniklasrose wants to merge 63 commits into
janniklasrose wants to merge 63 commits into
Conversation
janniklasrose
force-pushed
the
janniklasrose/grants-import-on-plan
branch
from
November 9, 2026 23:10
6e2c6db to
184364e
Compare
Contributor
|
Commit: Regression test reportSelected 20 changed tests (limit=20, 32 not selected) 44 tests: regression=40 unreleased=0 coverage=4 failing=0
TestAccept/bundle/resources/catalogs/rename/DMS=/READPLAN=: branch ✅ | main (1d69dfe) ❌ | latest (v1.26.0) ❌main (1d69dfe): latest (v1.26.0): TestAccept/bundle/resources/catalogs/rename/DMS=/READPLAN=1: branch ✅ | main (1d69dfe) ❌ | latest (v1.26.0) ❌main (1d69dfe): latest (v1.26.0): TestAccept/bundle/resources/catalogs/rename/DMS=true/READPLAN=: branch ✅ | main (1d69dfe) ❌ | latest (v1.26.0) ❌main (1d69dfe): latest (v1.26.0): 37 more failure details omitted, see the full report in the run. |
Collaborator
✅ Integration test reportCommit: 5dccd12
Top 30 slowest tests (at least 2 minutes):
|
janniklasrose
force-pushed
the
janniklasrose/grants-import-on-plan
branch
from
November 11, 2026 19:51
184364e to
5a3378f
Compare
Changing a schema's name used to delete and recreate it. It is now an updatable ID field renamed via PATCH new_name, which keeps the schema and everything in it. UC moves the children of a renamed catalog or schema with it. A child volume (or a schema in a renamed catalog) whose catalog_name/schema_name references the parent's name was planned as a recreate, and its create then failed with RESOURCE_ALREADY_EXISTS on every deploy. The planner now re-keys such children (update_id, reason parent_renamed), and they are addressed under their new parents. The test server now moves children on catalog/schema rename and rejects duplicate volume names, matching UC. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
A captured terraform fixture (migrate/command/basic) deploys the same volume name from two targets. The plan goldens already catch a regression to recreate, so the duplicate check is not needed. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Also restores the grants/schemas/change_name golden for the in-place rename, which the rebase onto denik/grants-rekey dropped. Co-authored-by: Isaac <no-reply@databricks.com>
Also restores the grants/schemas/change_name golden for the in-place rename. Co-authored-by: Isaac <no-reply@databricks.com>
schema-rename now renames in place, so it no longer covers the schema recreate. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…ename UC moves schemas and volumes with their renamed catalog or schema, so the re-key needs no update. Read the child instead to record its full name as UC spells it. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…ename Co-authored-by: Isaac <no-reply@databricks.com>
Azure and GCP return their own volume storage_location and predictive optimization metastore, and UC may touch a schema's updated_at after create. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Like the other DoUpdateWithID implementations, take the new ID from config, so a child moved by its parent's rename needs no request at all. Co-authored-by: Isaac <no-reply@databricks.com>
…lied A deploy can stop after renaming a schema but before re-keying its volume. The next plan then saw no rename in progress, read the volume at its old path, got 404, and planned a create, which fails on UC with RESOURCE_ALREADY_EXISTS. An update_id now records the old values of the fields it changes (renamed_from) in the same state write as the new ID. A child whose parent is not being renamed in the plan follows that recorded rename instead, and is not planned as a create just because its read at the old ID found nothing. Co-authored-by: Isaac <no-reply@databricks.com>
…amed_from When the read at the recorded ID finds nothing, the plan reads at the ID the config gives the resource and takes what it finds only if its stable ID (schema_id, volume_id, now saved in the state entry and the DMS envelope) matches. This covers a deploy that stopped after renaming the parent and a rename done out of band, and never adopts a different object of the same name. It replaces renamed_from, which missed repeated renames and could not tell a rename from a recreate. The test server now deletes children with a forced catalog/schema delete and rejects a catalog create under a taken name, like UC. The migrate fixtures dropped the out-of-band catalog create that the scripts already run. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…of stable_id The internal ID now lives in the volume and schema state (VolumeState, SchemaState), set from the API responses and copied by RemapState; the framework reads it back through a new optional InternalID method. This replaces the stable_id field on the state entry and in the DMS envelope. A new drop_changes lifecycle rule leaves the always-different volume_id and schema_id (config never sets them) out of the plan. Co-authored-by: Isaac <no-reply@databricks.com>
A parent renamed in the same plan moves the resource before its recreate runs. The recreate now deletes it at the ID the rename gives it, after the rename. The parent comes from the state's depends_on, since the config may no longer reference it. Co-authored-by: Isaac <no-reply@databricks.com>
The plan's state_only re-keys, found IDs and added depends_on must survive serialization. Co-authored-by: Isaac <no-reply@databricks.com>
Numbered UUIDs from main (#7021); restores the volume_id skip in change-name and the recorded schema_id in user_agent/simple, which the rebase dropped. Co-authored-by: Isaac <no-reply@databricks.com>
… case-insensitive volume lookups On AWS the re-keyed ID keeps the config's casing (main.s.MyVolume) and UC still reads the volume at it, so the next plan is a no-op. The test server now looks a volume's full name up case-insensitively too, instead of planning a create. Co-authored-by: Isaac <no-reply@databricks.com>
Without IDFromState the update_id would reach DoUpdateWithID with the stale ID. Co-authored-by: Isaac <no-reply@databricks.com>
…ew B5) Co-authored-by: Isaac <no-reply@databricks.com>
…ad (review B6) Co-authored-by: Isaac <no-reply@databricks.com>
…view B7) Co-authored-by: Isaac <no-reply@databricks.com>
…w B8) Co-authored-by: Isaac <no-reply@databricks.com>
…om four goldens Co-authored-by: Isaac <no-reply@databricks.com>
… rename (review A3, Badness); testserver: reject a taken schema name Recorded on AWS: the schema's create fails with SCHEMA_ALREADY_EXISTS. The test server now rejects it the same way instead of creating it again. Co-authored-by: Isaac <no-reply@databricks.com>
The test server now rejects a taken schema name, as UC does. Co-authored-by: Isaac <no-reply@databricks.com>
… A3) A state recorded without schema_id/volume_id (by an older CLI, or by bind) never got them, since a skip writes no state, so the plan could not follow an out-of-band rename. An otherwise unchanged resource whose state lacks the ID is now planned as a state-only update that records it from the remote state. Co-authored-by: Isaac <no-reply@databricks.com>
…y (review A3) The terraform state has no volume_id, and the migrated state would otherwise get the IDs from a state-only update in the next plan. The migration's plan check already read every resource, so record them from there. Co-authored-by: Isaac <no-reply@databricks.com>
The provider records UC's schema_id on databricks_schema. databricks_volume has no volume_id, so a volume migrated this way has none. Co-authored-by: Isaac <no-reply@databricks.com>
… requests The fixture was captured with the init script, so replaying it after the init created the source schema twice, which the test server now rejects as UC does. Co-authored-by: Isaac <no-reply@databricks.com>
It was captured with the init script's schema and source-table requests, which every test runs anyway, so the replay created the schema twice. Reverts the INVARIANT_REPLAY workaround. Co-authored-by: Isaac <no-reply@databricks.com>
The command plans nothing and the terraform state has no volume_id, so it reads each resource whose migrated state lacks its internal ID. A failed read only warns: the next deploy records the ID with a state-only update. Co-authored-by: Isaac <no-reply@databricks.com>
… saved plan too (READPLAN) Co-authored-by: Isaac <no-reply@databricks.com>
… recorded name (review B1, B2, Badness) Both gaps are on main too: destroy never looks for a moved resource, and the plan takes whatever is at the recorded ID without checking schema_id. Co-authored-by: Isaac <no-reply@databricks.com>
… Badness) Co-authored-by: Isaac <no-reply@databricks.com>
Before the lookup existed the plan created the resource; an error other than not-found at the config's ID now leads to the same plan, with a warning. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Use an optional resource adapter ImportID hook to compare new grants nodes with their current remote assignments. Keep planning read-only and persist adoption during apply, including matching grants and saved plans. Record imports as bind or bind-and-update operations in deployment history. Cover matching adoption, empty-list revocation, differing grants and extra principals, saved-plan deployment, and read failures with acceptance tests.
janniklasrose
force-pushed
the
janniklasrose/grants-import-on-plan
branch
from
November 11, 2026 22:19
c811e72 to
5dccd12
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Based on #7343. Read existing grants on the first plan using its
IDFromStatehook, with a grants-only adoption opt-in. Matching grants use a state-only update; differing grants include revocations. Apply records adoption for saved plans and deployment history.Why
Adding grants to an existing resource should show the actual changes. Sharing identity extraction and state-only writes avoids duplicate machinery while preserving schema and volume rename safeguards. Optional
DoCreatefollows in #7510.Tests
This PR was written with Codex.