Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .nextchanges/bundles/optional-create.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* Report grants and permissions as updates when their parent resources are first deployed. ([#7510](https://github.com/databricks/cli/pull/7510))
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
Created jobs.my_job
Created jobs.my_job.permissions
Updated jobs.my_job.permissions
Files: 2 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

=== Add a permission remotely (a change on the permissions sub-resource)

Expand Down
4 changes: 2 additions & 2 deletions acceptance/bundle/deploy/immutable/can-manage/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more

Created internal_immutable_snapshots.immutable
Created jobs.my_job
Created jobs.my_job.permissions
Updated jobs.my_job.permissions
Files: 0 uploaded, 0 deleted
Resources: 3 created, 0 changed, 0 deleted, 0 unchanged
Resources: 2 created, 1 changed, 0 deleted, 0 unchanged

=== Check can_manage_principals sent to the snapshot create API
>>> print_requests.py //api/2.0/snapshots
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ Consider using a adding a top-level permissions section such as the following:
See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
in databricks.yml:22:3

Created jobs.my_job.permissions
Recreated internal_immutable_snapshots.immutable
Updated jobs.my_job
Updated jobs.my_job.permissions
Files: 0 uploaded, 0 deleted
Resources: 2 created, 1 changed, 1 deleted, 0 unchanged
Resources: 1 created, 2 changed, 1 deleted, 0 unchanged
Second snapshot path: /Workspace/Users/[UUID][0]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID][2]/snapshot/files/src/main.py

=== Verify snapshot path changed after permissions change
Expand Down
10 changes: 5 additions & 5 deletions acceptance/bundle/deploy/mlops-stacks/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -83,15 +83,15 @@ Warning: unknown field: description

Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/project_name_[UNIQUE_NAME]/dev/files...
Created experiments.experiment
Created experiments.experiment.permissions
Created jobs.batch_inference_job
Created jobs.batch_inference_job.permissions
Created jobs.model_training_job
Created jobs.model_training_job.permissions
Created registered_models.model
Created registered_models.model.grants
Updated experiments.experiment.permissions
Updated jobs.batch_inference_job.permissions
Updated jobs.model_training_job.permissions
Updated registered_models.model.grants
Files: 26 uploaded, 0 deleted
Resources: 8 created, 0 changed, 0 deleted, 0 unchanged
Resources: 4 created, 4 changed, 0 deleted, 0 unchanged

>>> [CLI] bundle summary --var catalog_name=main -o json
Warning: expected a string value, found null
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...
Created schemas.myschema
Created schemas.myschema.grants
Updated schemas.myschema.grants
Files: 1 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

>>> print_requests.py //unity-catalog/permissions --sort
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
bundle:
name: update-without-create-$UNIQUE_NAME

resources:
schemas:
schema:
catalog_name: main
name: schema_$UNIQUE_NAME
grants:
- principal: ${workspace.current_user.userName}
privileges: [USE_SCHEMA]
jobs:
job:
name: job_$UNIQUE_NAME
tasks:
- task_key: main
notebook_task:
notebook_path: ./hello.py
permissions:
- service_principal_name: ${workspace.current_user.userName}
level: IS_OWNER
2 changes: 2 additions & 0 deletions acceptance/bundle/deploy/update-without-create/hello.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Databricks notebook source
print("hello")
14 changes: 14 additions & 0 deletions acceptance/bundle/deploy/update-without-create/out.actions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"resources.jobs.job": {
"action": "create"
},
"resources.jobs.job.permissions": {
"action": "update"
},
"resources.schemas.schema": {
"action": "create"
},
"resources.schemas.schema.grants": {
"action": "update"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Cloud = true
EnvMatrix.DMS = ["", "true"]
EnvMatrix.READPLAN = ["", "1"]
161 changes: 161 additions & 0 deletions acceptance/bundle/deploy/update-without-create/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@

=== Plan updates for grants and permissions before their parents exist
>>> [CLI] bundle plan
create jobs.job
update jobs.job.permissions
create schemas.schema
update schemas.schema.grants

Plan: 2 to add, 2 to change, 0 to delete, 0 unchanged

>>> [CLI] bundle plan -o json

>>> print_state.py

=== Reject old create actions before modifying resources
>>> musterr [CLI] bundle deploy --plan legacy-plan.json
Error: cannot create resources.jobs.job.permissions: resource does not support creation; run 'bundle plan' again


>>> musterr [CLI] bundle deploy --plan legacy-plan.json
Error: cannot recreate resources.jobs.job.permissions: resource does not support creation; run 'bundle plan' again


>>> print_requests.py //jobs //unity-catalog

>>> print_state.py

=== Create the parents and apply their dependent updates
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/update-without-create-[UNIQUE_NAME]/default/files...
Created jobs.job
Created schemas.schema
Updated jobs.job.permissions
Updated schemas.schema.grants
Files: 5 uploaded, 0 deleted
Resources: 2 created, 2 changed, 0 deleted, 0 unchanged

>>> print_requests.py //api/2.0/permissions/jobs //unity-catalog/permissions --sort
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/[SCHEMA_ID]",
"body": {
"changes": [
{
"add": [
"USE_SCHEMA"
],
"principal": "[USERNAME]",
"remove": [
"ALL_PRIVILEGES"
]
}
]
}
}
{
"method": "PUT",
"path": "/api/2.0/permissions/jobs/[JOB_ID]",
"body": {
"access_control_list": [
{
"permission_level": "IS_OWNER",
"service_principal_name": "[USERNAME]"
}
]
}
}

>>> print_state.py
{
"resources.jobs.job.permissions": {
"__id__": "/jobs/[JOB_ID]",
"state": {
"object_id": "/jobs/[JOB_ID]",
"__embed__": [
{
"level": "IS_OWNER",
"service_principal_name": "[USERNAME]"
}
]
},
"depends_on": [
{
"node": "resources.jobs.job",
"label": "${resources.jobs.job.id}"
}
]
},
"resources.schemas.schema.grants": {
"__id__": "schema/[SCHEMA_ID]",
"state": {
"securable_type": "schema",
"full_name": "[SCHEMA_ID]",
"__embed__": [
{
"principal": "[USERNAME]",
"privileges": [
"USE_SCHEMA"
]
}
]
},
"depends_on": [
{
"node": "resources.schemas.schema",
"label": "${resources.schemas.schema.id}"
}
]
}
}

=== A second plan is unchanged
>>> [CLI] bundle plan
Plan: 0 to add, 0 to change, 0 to delete, 4 unchanged

=== Recreate a deleted parent and move its permissions to the new ID
>>> [CLI] jobs delete [JOB_ID]

>>> [CLI] bundle plan
create jobs.job
update jobs.job.permissions

Plan: 1 to add, 1 to change, 0 to delete, 2 unchanged

>>> [CLI] bundle plan -o json

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/update-without-create-[UNIQUE_NAME]/default/files...
Created jobs.job
Updated jobs.job.permissions
Files: 1 uploaded, 0 deleted
Resources: 1 created, 1 changed, 0 deleted, 2 unchanged

>>> print_requests.py //api/2.0/permissions/jobs
{
"method": "PUT",
"path": "/api/2.0/permissions/jobs/[JOB_ID_4]",
"body": {
"access_control_list": [
{
"permission_level": "IS_OWNER",
"service_principal_name": "[USERNAME]"
}
]
}
}

>>> [CLI] bundle plan
Plan: 0 to add, 0 to change, 0 to delete, 4 unchanged

>>> errcode [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.jobs.job
delete resources.schemas.schema

This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
delete resources.schemas.schema

All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/update-without-create-[UNIQUE_NAME]/default

Destroy: 2 deleted
41 changes: 41 additions & 0 deletions acceptance/bundle/deploy/update-without-create/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
envsubst < databricks.yml.tmpl > databricks.yml

cleanup() {
trace errcode $CLI bundle destroy --auto-approve
rm -f "$OUT_REQUESTS"
}
trap cleanup EXIT

title "Plan updates for grants and permissions before their parents exist"
trace $CLI bundle plan
trace $CLI bundle plan -o json > plan.json
jq '.plan | with_entries(.value |= {action})' plan.json > out.actions.json
trace print_state.py

title "Reject old create actions before modifying resources"
for action in create recreate; do
jq --arg action "$action" '.plan["resources.jobs.job.permissions"].action = $action' plan.json > legacy-plan.json
trace musterr $CLI bundle deploy --plan legacy-plan.json
done
rm legacy-plan.json
trace print_requests.py //jobs //unity-catalog
trace print_state.py

title "Create the parents and apply their dependent updates"
trace $CLI bundle deploy $(readplanarg plan.json)
replace_ids.py
trace print_requests.py //api/2.0/permissions/jobs //unity-catalog/permissions --sort
trace print_state.py | jq '.state | with_entries(select(.key | endswith(".permissions") or endswith(".grants")))'

title "A second plan is unchanged"
trace $CLI bundle plan

title "Recreate a deleted parent and move its permissions to the new ID"
job_id="$(read_id.py job)"
trace $CLI jobs delete "$job_id"
trace $CLI bundle plan
trace $CLI bundle plan -o json > plan.json
trace $CLI bundle deploy $(readplanarg plan.json)
replace_ids.py
trace print_requests.py //api/2.0/permissions/jobs
trace $CLI bundle plan
9 changes: 9 additions & 0 deletions acceptance/bundle/deploy/update-without-create/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Cloud = true
IsServicePrincipal = true
RecordRequests = true
EnvMatrix.READPLAN = ["", "1"]
Ignore = [".databricks", "databricks.yml", "plan.json"]

[[Repls]]
Old = " --plan plan.json"
New = ""
8 changes: 4 additions & 4 deletions acceptance/bundle/deployment/unbind/grants/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
Created schemas.schema_1
Created schemas.schema_1.grants
Updated schemas.schema_1.grants
Files: 2 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

>>> [CLI] grants get schema main.test-schema-[UNIQUE_NAME] --output json
{
Expand All @@ -20,9 +20,9 @@ Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
>>> [CLI] bundle deploy --var suffix=another
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
Created schemas.schema_1
Created schemas.schema_1.grants
Updated schemas.schema_1.grants
Files: 0 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

=== Grants should be the same as before unbind
>>> [CLI] grants get schema main.test-schema-[UNIQUE_NAME] --output json
Expand Down
8 changes: 4 additions & 4 deletions acceptance/bundle/deployment/unbind/permissions/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
Created jobs.job_1
Created jobs.job_1.permissions
Updated jobs.job_1.permissions
Files: 2 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

>>> [CLI] jobs get-permissions [JOB_1_ID] --output json
{
Expand All @@ -22,9 +22,9 @@ Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
Created jobs.job_1
Created jobs.job_1.permissions
Updated jobs.job_1.permissions
Files: 0 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

=== Permissions should be the same as before unbind
>>> [CLI] jobs get-permissions [JOB_1_ID] --output json
Expand Down
6 changes: 3 additions & 3 deletions acceptance/bundle/dms/empty-grants/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-empty-grants-[UNIQUE_NAME]/default/files...
Created schemas.foo
Created schemas.foo.grants
Updated schemas.foo.grants
Files: 2 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
Resources: 1 created, 1 changed, 0 deleted, 0 unchanged

>>> print_requests.py --dms //api/2.0/bundle
{
Expand Down Expand Up @@ -35,7 +35,7 @@ Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
"resource_key": "schemas.foo"
},
{
"action_type": "OPERATION_ACTION_TYPE_CREATE",
"action_type": "OPERATION_ACTION_TYPE_UPDATE",
"resource_key": "schemas.foo.grants"
}
],
Expand Down
Loading
Loading