Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .nextchanges/bundles/5183.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* Add `isolation_mode` to the `catalogs` resource, so a catalog can be deployed as `OPEN` or `ISOLATED`. ([#7463](https://github.com/databricks/cli/pull/7463))
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ resources:
name: test-catalog-$UNIQUE_NAME
comment: This is a test catalog
custom_max_retention_hours: 48 # CAN_REMOVE
isolation_mode: ISOLATED # CAN_REMOVE
managed_encryption_settings:
customer_managed_key_id: 00000000-0000-0000-0000-000000000000
properties:
Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/refschema/output/catalogs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ resources.catalogs.*.effective_predictive_optimization_flag.value catalog.Enable
resources.catalogs.*.enable_predictive_optimization catalog.EnablePredictiveOptimization REMOTE
resources.catalogs.*.full_name string REMOTE
resources.catalogs.*.id string INPUT
resources.catalogs.*.isolation_mode catalog.CatalogIsolationMode REMOTE
resources.catalogs.*.isolation_mode catalog.CatalogIsolationMode ALL
resources.catalogs.*.lifecycle resources.Lifecycle INPUT
resources.catalogs.*.lifecycle.depends_on []resources.DependsOn INPUT
resources.catalogs.*.lifecycle.depends_on[*] resources.DependsOn INPUT
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
"comment": {
"action": "update",
"remote": "set outside the bundle"
},
"isolation_mode": {
"action": "skip",
"reason": "backend_default",
"remote": "OPEN"
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged

>>> [CLI] bundle plan --output json
{
"isolation_mode": {
"action": "skip",
"reason": "backend_default",
"remote": "OPEN"
},
"properties": {
"action": "skip",
"reason": "backend_default",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
bundle:
name: test-bundle-$UNIQUE_NAME

resources:
catalogs:
catalog1:
name: test-catalog-$UNIQUE_NAME
isolation_mode: ISOLATED
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Cloud = false
EnvMatrix.DMS = ["", "true"]
123 changes: 123 additions & 0 deletions acceptance/bundle/resources/catalogs/isolation_mode/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@

=== Validate accepts isolation_mode

>>> [CLI] bundle validate --output json
{
"name": "test-catalog-[UNIQUE_NAME]",
"isolation_mode": "ISOLATED"
}

=== Deploy: CreateCatalog does not accept isolation_mode, so the catalog is created and then patched

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
Created catalogs.catalog1
Files: 2 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged

>>> print_requests.py //unity-catalog/catalogs
{
"method": "POST",
"path": "/api/2.1/unity-catalog/catalogs",
"body": {
"name": "test-catalog-[UNIQUE_NAME]"
}
}
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/catalogs/test-catalog-[UNIQUE_NAME]",
"body": {
"isolation_mode": "ISOLATED"
}
}

>>> [CLI] catalogs get test-catalog-[UNIQUE_NAME]
{
"name": "test-catalog-[UNIQUE_NAME]",
"isolation_mode": "ISOLATED"
}

=== Redeploy is a no-op

>>> [CLI] bundle plan
Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged

=== Removing isolation_mode from the config leaves the remote value alone

>>> update_file.py databricks.yml isolation_mode: ISOLATED # isolation_mode removed

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
Updated catalogs.catalog1
Files: 1 uploaded, 0 deleted
Resources: 0 created, 1 changed, 0 deleted, 0 unchanged

>>> print_requests.py //unity-catalog/catalogs
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/catalogs/test-catalog-[UNIQUE_NAME]",
"body": {
"comment": "",
"custom_max_retention_hours": 0
}
}

>>> [CLI] catalogs get test-catalog-[UNIQUE_NAME]
{
"name": "test-catalog-[UNIQUE_NAME]",
"isolation_mode": "ISOLATED"
}

>>> [CLI] bundle plan --output json
{
"comment": {
"action": "skip",
"reason": "empty",
"remote": ""
},
"custom_max_retention_hours": {
"action": "skip",
"reason": "empty",
"remote": 0
},
"isolation_mode": {
"action": "skip",
"reason": "backend_default",
"remote": "ISOLATED"
}
}

=== Setting OPEN patches the catalog

>>> update_file.py databricks.yml # isolation_mode removed isolation_mode: OPEN

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
Updated catalogs.catalog1
Files: 1 uploaded, 0 deleted
Resources: 0 created, 1 changed, 0 deleted, 0 unchanged

>>> print_requests.py //unity-catalog/catalogs
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/catalogs/test-catalog-[UNIQUE_NAME]",
"body": {
"comment": "",
"custom_max_retention_hours": 0,
"isolation_mode": "OPEN"
}
}

>>> [CLI] catalogs get test-catalog-[UNIQUE_NAME]
{
"name": "test-catalog-[UNIQUE_NAME]",
"isolation_mode": "OPEN"
}

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.catalogs.catalog1

All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default

Destroy: 1 deleted
35 changes: 35 additions & 0 deletions acceptance/bundle/resources/catalogs/isolation_mode/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
envsubst < databricks.yml.tmpl > databricks.yml

CATALOG="test-catalog-${UNIQUE_NAME}"

cleanup() {
trace $CLI bundle destroy --auto-approve

# Consume the destroy's requests so none are left recorded.
print_requests.py //unity-catalog/catalogs &> LOG.destroy_requests
}
trap cleanup EXIT

title "Validate accepts isolation_mode"
trace $CLI bundle validate --output json | jq '.resources.catalogs.catalog1 | {name, isolation_mode}'

title "Deploy: CreateCatalog does not accept isolation_mode, so the catalog is created and then patched"
trace $CLI bundle deploy
trace print_requests.py //unity-catalog/catalogs
trace $CLI catalogs get "$CATALOG" | jq '{name, isolation_mode}'

title "Redeploy is a no-op"
trace $CLI bundle plan

title "Removing isolation_mode from the config leaves the remote value alone"
trace update_file.py databricks.yml "isolation_mode: ISOLATED" "# isolation_mode removed"
trace $CLI bundle deploy
trace print_requests.py //unity-catalog/catalogs
trace $CLI catalogs get "$CATALOG" | jq '{name, isolation_mode}'
trace $CLI bundle plan --output json | nostamp | jq '.plan[].changes'

title "Setting OPEN patches the catalog"
trace update_file.py databricks.yml "# isolation_mode removed" "isolation_mode: OPEN"
trace $CLI bundle deploy
trace print_requests.py //unity-catalog/catalogs
trace $CLI catalogs get "$CATALOG" | jq '{name, isolation_mode}'
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Local only: the create-then-update sequence is direct-engine-specific, and whether an
# ISOLATED catalog stays readable from the deploying workspace without an explicit
# workspace binding has not been verified against a real workspace.

Ignore = [
".databricks",
"databricks.yml",
]
4 changes: 4 additions & 0 deletions bundle/config/resources/catalog.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ type Catalog struct {
ID string `json:"id,omitempty" bundle:"readonly"`
catalog.CreateCatalog

// IsolationMode is declared here rather than taken from CreateCatalog because the
// Create API does not accept it; UC only takes it on update.
IsolationMode catalog.CatalogIsolationMode `json:"isolation_mode,omitempty"`

// List of grants to apply on this catalog.
Grants []catalog.PrivilegeAssignment `json:"grants,omitempty"`
}
Expand Down
7 changes: 5 additions & 2 deletions bundle/direct/dresources/all_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,11 @@ var testConfig map[string]any = map[string]any{
"catalogs": &resources.Catalog{
Name: "mycatalog",
Comment: "Test catalog",
// Note: EnablePredictiveOptimization and IsolationMode cannot be set during creation,
// only during updates. They are not included in the test config.
// Note: EnablePredictiveOptimization cannot be set during creation,
// only during updates. It is not included in the test config.
// IsolationMode has the same restriction but is applied by DoCreate with a
// follow-up update, so it is set here to cover that path.
IsolationMode: catalog.CatalogIsolationModeIsolated,
},

"external_locations": &resources.ExternalLocation{
Expand Down
2 changes: 1 addition & 1 deletion bundle/direct/dresources/apitypes.generated.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading