Repository navigation
fix(usage): load the Usage report from the shared aggregate, fit the models table, name apps - #8728
Closed
andrew05060414 wants to merge 7 commits into
Closed
andrew05060414 wants to merge 7 commits into
andrew05060414 wants to merge 7 commits into
Conversation
…xis's source, and probe what a relay accepts GET /api/models routed rows gain capabilitySources (manual > probed > reported > inferred > unknown) and display-only inferredCapabilities from a same-name lookup over the vendor registry, bundled metadata and other providers' declarations. POST /api/providers/probe-capabilities sends a baseline, a 1x1 image and one request per effort through the completion diagnostic, asks before spending on a metered provider, and with apply writes the conclusive axes plus a sparse modelCapabilityProbes label.
…el every price's source Automatic prices fill the two lowest tiers of price resolution (manual > official > bundled > models.dev > OpenRouter). Catalogs are cached in price-catalog-cache.json, refreshed daily and on a pricing miss (merging only the missing keys, 6h per-key cooldown, never on the request path). Catalog prices are verified-derived estimates carrying their feed URL and fetch time. Adds an opt-in providers.<name>.priceMultiplier for relays (never applied to manual prices), apiEquivalent labelling for subscription providers, a price field on /api/models rows, POST /api/pricing/refresh and PUT /api/providers/:name/price-multiplier. The dashboard shows a source chip per model and adds adopt-as-manual, refresh and multiplier controls. Also registers POST /api/providers/probe-capabilities, which the base commit left out of the route registry.
…e route registry The capability probe handler was routed in provider-routes but never declared, so the registry reconciliation test failed.
…name its apps The dashboard always asks for client=all&device=all. The route treated that as a filtered query, so it skipped the warmed range:surface cache and rebuilt the report through the filter index, which re-reads every row by offset. On a 270k-row ledger the cold request took 24-65 s and the page hit its 30 s timeout. Every-source queries now read the unfiltered aggregate and echo the source fields the page validates. The models table fits 1500 px in seven columns, with secondary figures as a second line. Apps rows written before client tracking are split by the wire they used, unknown clients record a bounded product name from their originator or User-Agent, and the excluded-request caption and the coverage panel explain themselves on hover.
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Contributor
|
Contributor
⏳ DRAFT
What to do
Review readiness checklist
✅ 4/4 boxes ticked. This pull request was already a draft. Its draft status will be preserved after every issue above is resolved. Hygiene
|
Contributor
Author
|
Superseded by #8738, which carries this change together with the related model metadata and usage PRs as one commit each on current |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem. On a busy installation the Usage & Logs page was slow enough to fail. With a synthetic ledger shaped like a real one (270k rows, 745 MB, ~135k requests in 30 days) the dashboard's report request took 23.8 to 64.5 s cold, past the page's 30 s resource timeout, so the page showed "resource request timed out after 30000ms". The models table also needed horizontal scrolling at 1500 px, the Apps table lumped most traffic into "Unclassified" and "Unknown" with no explanation, and the "N requests excluded" captions and the coverage figure gave no reason.
After this PR.
curl,multica-agent-sdk,claude-agent-sdkfor the Claude Agent SDK) and show as their own rows. A note under the table says what the two buckets mean.Root cause of the slowness. The dashboard always sends
client=all&device=all. The route treated that as a filtered query, so it bypassed the warmedrange:surfacecache and went through the filter index, which re-reads every matching row by offset (all of them, for this filter) and builds a 356 MB SQLite projection. It also never shared work with the Home page's unfiltered request.client=all&device=allselects every row, so it is now served from the unfiltered aggregate and echoed back with thesourceFilterandlegacyUnclassifiedRequestsfields the page validates.Implementation notes.
src/server/management/usage-source-presentation.ts:selectsEverySourceand the response shaping; the legacy (no-source) response keeps its old shape.src/usage/client.ts:usageClientNamekeeps only the first product token oforiginator/User-Agent, lowercased, versionless, matching^[a-z0-9][a-z0-9._-]{0,39}$, or nothing. It is recorded only when the closed classifier saysunknown; header values are never stored.src/usage/app-breakdown.ts: apps rows gain optionalname/legacyWire; at most 24 names per report (and a per-partition key budget), the rest fold into plain Unknown. The hub schema accepts the new fields.usageApp, source filters and the SQL filter index is unchanged.Stacked on #8708 (auto model pricing) because the hover copy and the base branch's price catalog belong together; it contains that PR's commits until it lands.
Verification
Measured on a local instance (
OPENCODEX_HOMEunder.tmp/) seeded with generated rows; no real usage data was used.GET /api/usage?range=30d&client=all&device=allcoldbun run typecheck,bun run lint:gui,bun run privacy:scan,bun run structure:check,bun run skill:surface:check: pass.bun test tests/ci-workflows/file-size-ratchet.test.ts: 9 pass.bun test tests/usage/usage-client-attribution.test.ts: 23 pass (new: client naming, legacy-wire split, name folding).bun test tests/server/hub-usage.test.ts: 7 pass.usage-layout,usage-custom-range,usage-app-breakdownand the othergui/tests/*usage*files: 241 pass after updating the two table-layout assertions; new case for named rows, legacy wires and the exclusion tooltip.tests/server/api-usage.test.tsandtests/server/api-key-attribution.test.tscould not be run to completion on the Windows dev machine: every server-starting case fails inafterEachwithEBUSY ... rm <tmp>while the config-dir ACL hardening (icacls) is still running, including cases this PR does not touch. The new route case (the all-apps, all-devices dashboard query reuses the unfiltered aggregate) is left to CI; please run withci:full.Screenshots (synthetic data):
English, 1500 px wide: models table without horizontal scroll, named apps, coverage note
Chinese
Readiness (2026-11-13)
dev(873ec9b), inside the gate's window; no rebase for review.bun run typecheck,structure:check,skill:surface:check,privacy:scan, the layout and file-size guards and this PR's focused tests; after the deploy one real request each through Claude, GPT, Gemini, DeepSeek and Grok returned 200 and the error rate did not rise.maintainer-sponsoredlabel: the hygiene gate flagssrc/server/auth-cors.ts(through feat(pricing): fetch model prices from models.dev and OpenRouter, label every price's source #8708, which this PR is stacked on). Everything else is ready for review.Checklist
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
Required local validation passed; commands, results, and any full-suite exception are documented.
I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.