Repository navigation
Add a hint when an app's service principal is missing during bundle deploy - #7533
Conversation
…eploy Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/bundle/resources/apps/missing-service-principal/DMS=: branch ✅ | main (634e6bf) ❌ | latest (v1.26.0) ❌main (634e6bf): latest (v1.26.0): TestAccept/bundle/resources/apps/missing-service-principal/DMS=true: branch ✅ | main (634e6bf) ❌ | latest (v1.26.0) ❌main (634e6bf): latest (v1.26.0): |
✅ Integration test reportCommit: bea51a5
Top 13 slowest tests (at least 2 minutes):
|
aws: TestAccept/bundle/resources/apps/missing-service-principal Co-authored-by: Isaac <no-reply@databricks.com>
| return err | ||
| } | ||
| return fmt.Errorf("%w\n\nThe app cannot be updated or started without its service principal. "+ | ||
| "To recreate the app, delete it with \"databricks apps delete %s\", wait for the deletion to finish, "+ |
There was a problem hiding this comment.
DoCreate already retries while the app is DELETING (app.go:143) and IsGone plans a create for it, so "wait for the deletion to finish" isn't needed. Drop that clause?
There was a problem hiding this comment.
I think the clause is still needed. IsGone is only checked for resources being deleted (bundle_plan.go, delete branch, and deleteConfirmedGone in apply.go). For an app still in config, the planner reads the app, and while it is DELETING, GET still returns it. A redeploy then plans an update, not a create, and the DoCreate DELETING retry is never reached. I checked on the testserver: bundle deploy, then apps delete myappname, then bundle plan shows update apps.mykey (Plan: 0 to add, 1 to change). A create is planned only after GET returns 404, which means after the deletion finishes. So I left the hint as is in the follow-up #7545. Happy to drop the clause if you would rather handle DELETING in the planner instead.
AI agent (harness_auto, opus5.5, isaac_claude)
| @@ -0,0 +1,13 @@ | |||
| trace $CLI bundle deploy | |||
| rm -f out.requests.txt | |||
There was a problem hiding this comment.
This rm -f is a no-op: the harness records to $OUT_REQUESTS in a sibling dir (acceptance_test.go:913), so the first print_requests also lists the create. Drop the line?
There was a problem hiding this comment.
Agreed, thanks. It is a no-op; removed in #7545 (the golden output is unchanged, since the first print_requests already listed the create).
AI agent (harness_auto, opus5.5, isaac_claude)
## Changes Follow-up to [databricks#7533](databricks#7533), addressing review feedback: remove `rm -f out.requests.txt` from the `bundle/resources/apps/missing-service-principal` script. The harness records requests to `$OUT_REQUESTS` in a separate output dir, so the line did nothing. The golden output is unchanged: the first `print_requests.py` already listed the create request. The hint text from databricks#7533 stays as is ("wait for the deletion to finish"). Right after `databricks apps delete`, the app is still returned in DELETING, and `bundle plan` plans an update, not a create. `IsGone` is only checked for resources being deleted. So a redeploy run before the deletion finishes does not recreate the app. Details are in the review thread on databricks#7533. ## Tests `go test ./acceptance -run TestAccept/bundle/resources/apps/missing-service-principal` passes locally with no output change. _Written by an AI agent (harness_auto, opus5.5, isaac_claude)._ This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
Changes
When
bundle deployupdates or starts an app whose service principal is gone, the Apps backend returns a 400 that says to delete the app. The CLI now appends an actionable hint to that error: delete the app withdatabricks apps delete <name>, wait for the deletion to finish, and deploy again to recreate it.The hint is added once at the top of the direct engine's app
DoUpdate(covering the update call, a non-SUCCEEDED update status, and the start/deploy lifecycle step) andWaitAfterCreate. It matches the substringservice principal does not exist or is deletedwithout regard to case, which is present in both backend variants (the update/start 400 and the compute ERROR status message). The original error and its API details are preserved (wrapped with%w); nothing else changes.Why
These failures come from a backend issue, not the CLI. Once an app hits this state, every later update or start on it fails until the app is recreated, so a retry would not help. The hint tells users what to do. It does not fix the flaky cloud tests that hit this backend issue; those are already listed in known_failures.
Judgment calls:
databricks apps deleteand a redeploy rather thanbundle destroy, because destroy removes every resource in the bundle.BAD_REQUESTor as a compute ERROR status message, so there is no error code or sentinel to match. If the wording changes, the only effect is that the hint stops appearing.(400 BAD_REQUEST)suffix appears after the hint text. Changing the error renderer would be a separate change.bundle deploy(the direct engine), as requested.bundle runanddatabricks apps startare not changed.Tests
New local acceptance test
bundle/resources/apps/missing-service-principal: the testserver returns the backend's 400 forPOST /api/2.0/apps/{name}/updateand/start. The test checks that the hint appears in both paths. A unit test covers the matcher: the API error, a wrapped error, and the compute ERROR status variant.Dashboard
Error: cannot update resources.apps.[NAME]: updating id=[UNIQUE_NAME]: App: [UNIQUE_NAME]'s service principal does not exist or is deleted. App cannot run without a valid service principal. Please delete your app. (400 BAD_REQUEST)Related, same backend symptom (hint applies too):
Error: cannot create resources.apps.[NAME]: waiting after creating id=[UNIQUE_NAME]: App: [UNIQUE_NAME]'s service principal does not exist or is deleted. ...Error: cannot update resources.apps.[NAME]: updating id=[UNIQUE_NAME]: failed to reach ACTIVE, got ERROR: App service principal does not exist or is deleted, application cannot run without a valid service principal. Please delete the app and re-create it.Written by an AI agent (harness_auto, opus5.5, isaac_claude).
This pull request and its description were written by Isaac.