Repository navigation
acc: don't record quality monitor status and dashboard_id - #7495
Conversation
Right after a monitor is created, the backend can still report MONITOR_STATUS_PENDING without a dashboard_id, which made the quality_monitors acceptance tests flaky on cloud. Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 4 changed tests (limit=20, 0 not selected) 8 tests: regression=0 unreleased=0 coverage=8 failing=0
|
✅ Integration test reportCommit: a9e4d50
Top 17 slowest tests (at least 2 minutes):
|
andrewnester
left a comment
There was a problem hiding this comment.
The integration report for this commit still marks change_table_name flaky on aws-windows (both variants) and azure-windows DMS=true. Was that attempt the same status/dashboard_id diff, or a different cause?
…-list Replace the jq del() deny-list with an allow-list helper in quality_monitors/script.prepare, as .agents/rules/testing.md asks, and reuse it for remote_state in the plan goldens. Co-authored-by: Isaac <no-reply@databricks.com>
|
AI agent (harness_auto, opus5.5, isaac_claude): Re the review question ("Was that attempt the same status/dashboard_id diff, or a different cause?"): it was a different cause, unrelated to this change. In all three flaky attempts on that run (aws-windows On azure-windows the timed-out file was |
Changes
Project quality monitor GET responses and plan
remote_statethrough amonitor_fields()allow-list inacceptance/bundle/resources/quality_monitors/script.prepare. This follows.agents/rules/testing.mdand matchespostgres_*/script.prepare. The allow-list leaves outstatusanddashboard_id. It applies tocreate/out.plan_noop.json,change_*/out.plan.jsonandchange_table_name/out.get.json. Thedashboard_idrepl intest.tomlhad nothing left to match, so I removed it.Why
These tests run
bundle plan/quality-monitors getright after the monitor is created or recreated. When that happens, the backend sometimes still reportsMONITOR_STATUS_PENDINGand hasn't setdashboard_idyet. The golden then doesn't match, and the tests are flaky on AWS, Azure and GCP.Decision: record less instead of polling until the monitor is
ACTIVE. Both fields are output-only and change on the server's schedule. None of these tests is about them: they cover plan actions, requests and state, and those are all still recorded. Polling would make the tests slower and would depend on how long provisioning takes on the backend.Dashboard:
"status": "MONITOR_STATUS_PENDING",(expected"status": "MONITOR_STATUS_ACTIVE",,"dashboard_id"missing) increate/out.plan_noop.jsonandchange_table_name/out.get.json--- .../out.plan.json- "dashboard_id": "(redacted)",- "status": "MONITOR_STATUS_ACTIVE",+ "status": "MONITOR_STATUS_PENDING",Tests: TestAccept/bundle/resources/quality_monitors/create/DMS=, TestAccept/bundle/resources/quality_monitors/change_table_name/DMS=, TestAccept/bundle/resources/quality_monitors/change_table_name/DMS=true (also
change_assets_dirandchange_output_schema_name).Tests
go test ./acceptance -run TestAccept/bundle/resources/quality_monitorspasses.quality_monitorssubtests pass on AWS (-count=1), both before and after the switch to the allow-list. Azure and GCP credentials were not available locally, so those clouds are left to CI.This pull request and its description were written by Isaac.