Repository navigation
air list: cache terminal runs' MLflow links on the Jobs scan path - #7627
Open
stevenchen-db wants to merge 2 commits into
Open
stevenchen-db wants to merge 2 commits into
stevenchen-db wants to merge 2 commits into
Conversation
The Jobs scan (default list, --all-users, user= filters) builds its rows from runs/list on every call and re-resolves each row's MLflow link with a get-output and an MLflow request, about 40 requests for 20 rows. A terminal run's link does not change, so cache it, like the index path already does for its rows. The scan uses its own cache component holding only the MLflow fields, so its entries never stand in for the index path's full rows. An entry is used only while the run's task runs and attempts match the ones it was resolved from, so a run repaired after it was cached is refetched. Active runs are never cached. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
✅ Integration test reportCommit: 38e141a
Top 3 slowest tests (at least 2 minutes):
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
air listmodes served by the Jobs scan (--all-users,--filter user=…, and the default active list) build their rows fromruns/liston every call and re-resolve each row's MLflow link with aget-outputand an MLflow request. A terminal run's link never changes, so it is now cached on disk, like the index path (--all-statusfor your own runs) already caches its rows.Why
On df1,
air list --all-users --all-statusandair list --all-status --filter user=<me>go from ~2.0s to ~0.6s once warm (20 terminal rows, ~40 requests skipped). The first call is unchanged.Tests
TestJobsScanCachesTerminalMLflowLinks: a terminal run is resolved once and then served from the cache; a repaired run (new attempt) is refetched; an active run is never cached; nothing is written to the index path's cache. Removing the cache hook or the task-run check each fails it.go test ./cmd/air(list tests also with-race),TestAccept/air, lint pass.--all-status, JSON, and the default list are identical to main.This pull request and its description were written by Isaac.