diff --git a/docs-site/src/content/docs/guides/web-dashboard.md b/docs-site/src/content/docs/guides/web-dashboard.md index ac68dae8d42..1d5f445ed2d 100644 --- a/docs-site/src/content/docs/guides/web-dashboard.md +++ b/docs-site/src/content/docs/guides/web-dashboard.md @@ -493,6 +493,8 @@ maintainers do not provide policy advice and cannot resolve provider enforcement measured usage. Active user `modelCosts` overlays take priority over bundled verified catalog and price fallbacks, and historical usage is re-estimated from the pricing active when the summary is read. The cost is an estimate for reconciliation, not a ChatGPT Plus/Pro subscription invoice. + A model with no bundled price is priced from the public models.dev or OpenRouter list and marked as + an online-list estimate; a manual price always wins. Historical bare `openai` rows that predate explicit attribution remain ambiguous rather than being assigned to the current main account. - **Target a specific Codex account from the model picker** is an explicit opt-in. When enabled, diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 40848033f42..24da1ca1e74 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -374,6 +374,7 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity | `defaultMaxOutputTokens?` | `number` | Provider-wide `openai-chat` fallback when the client omits `max_output_tokens`. An `anthropic` row without it uses the registry output cap of the Claude model it serves (64000 for any model on `api.anthropic.com`), and 8192 for a model id the registry does not list. A value you set here, or in `modelMaxOutputTokens`, always wins. | | `modelMaxOutputTokens?` | `Record` | Positive per-model `openai-chat` fallback budgets; exact/pattern matches beat the provider default. | | `modelCosts?` | `Record` | Per-model display prices (USD per 1M tokens), keyed by that provider's exact upstream model id — not a provider identifier or a routed `provider/model` label, e.g. `{ "deepseek-v4-flash": { "input": 0.14, "output": 0.28, "cacheRead": 0.0028, "cacheWrite": 0 } }`. Any model id is a valid key — custom providers may target any OpenAI-compatible endpoint through the `openai-chat` adapter, and local or internal provider ids work even when they are absent from the built-in catalogs. User-configured prices win over the built-in catalogs in the Logs `~$` and Usage estimates; historical entries are repriced from the current overlay, so editing a price can move past totals. The fallback order is user `modelCosts` → exact official correction → jawcode catalog → expected-price overlay → model-level vendor fallback, and an explicit all-zero user entry means a known-zero estimate; delete that model entry to restore automatic pricing. All-zero catalog metadata still falls through. Each rate must be a non-negative finite number at most 1,000,000 (USD per 1M tokens); out-of-range rows are rejected by the management boundary and dropped on load. Display-time estimation only: overlays never affect routing, account selection, quotas, or billing. | +| `priceMultiplier?` | `number` | Opt-in relay multiplier (finite, greater than 0, at most 1000) applied to this provider's automatic prices (official, models.dev, OpenRouter) in the cost estimate, for a relay that marks prices up or down. Never applied to `modelCosts`. Absent means 1. | | `headers?` | `Record` | Extra upstream headers. For the classic OpenAI web-search sidecar, `x-openai-internal-codex-responses-lite` is removed case-insensitively after configured and selected headers are merged; other forwarded headers are preserved. Authorization, cookies, API-key headers, embedded newlines, and invalid names are rejected. Requests that ride the provider outbound wrapper — model discovery, connection tests, and other proxy-originated diagnostics such as the Ollama show probe — send `User-Agent: opencodex` unless a User-Agent is set here or by the provider preset. An experimental relay may negotiate earlier completed-tool delivery with `x-opencodex-tool-completion: block-stop-v1`; the embedding shell must supply authenticated durable turn membership, and the relay must acknowledge the same response header and implement exact call-bound completion markers. Leave it unset for other providers. Early delivery applies only to eligible structured calls; later errors or cancellation cannot retract a call already delivered. | | `forwardClientHeaders?` | `string[]` | `openai-responses` only: opt in to copying named inbound client metadata headers onto inference requests when the provider has not already set that header. Credential and transport-owned names such as `Authorization`, cookies, API-key headers, `Content-Type`, `Content-Length`, `Host`, and `x-oai-attestation` are rejected and also filtered at runtime. Header names are case-insensitive and may not repeat. `headers` remains authoritative; the existing caller `User-Agent` fallback still applies when neither surface sets one. Supported names: `originator`, `x-client-request-id`, `x-codex-app-version`, `user-agent`, `x-codex-window-id`, `x-codex-installation-id`; all other names are refused. | | `supportsNativeCompactionTrigger?` | `boolean` | `openai-responses` only: declare support for native v2 `compaction_trigger` input items. Defaults off for noncanonical gateways; preserves native compaction rather than requesting a portable text summary. Independent of the v1 endpoint and blob replay capabilities. | @@ -1837,6 +1838,8 @@ contracts. The raw provider editor and provider API expose this map. POST/PUT replace an explicitly supplied map and reject null entries. PATCH merges individual axes; null clears a map, model, axis or video processing value, while `{}` makes no change. Omitted provider overwrites preserve the existing map. Malformed hand-edited files retain valid independent axes and treat malformed explicit input modalities as text-only, with a diagnostic. +`modelCapabilityProbes` is written by the [capability probe](/reference/management-api/#capability-probe), not by hand. Keyed by model ID, each entry records `probedAt` (ISO 8601 UTC) and `inputModalities: true` and/or `reasoningEfforts: true` for the declarations (`modelCapabilities..inputModalities`, `modelReasoningEfforts.`) that came from a live probe. A manual edit of that declaration through the dashboard or management API removes its label, a label is removed when its declaration is gone, and the field is absent when empty. A malformed value is dropped at load with a warning. + An explicit `modelCapabilities..inputModalities` now takes precedence over legacy modality hints for that exact routed model. A text-only declaration uses the existing vision sidecar to replace images with descriptions; if no sidecar is available, the request receives an explicit omission marker before dispatch. Native Chat image requests divert through this path. The catalog can still advertise image attachment support because the proxy provides the description step. Context-tier and video processing declarations remain inert pending their transport support. An input-modality declaration on a **custom model row** — the Input modalities checkboxes in the diff --git a/docs-site/src/content/docs/reference/management-api.md b/docs-site/src/content/docs/reference/management-api.md index 53f32c3e469..22c9147fcf7 100644 --- a/docs-site/src/content/docs/reference/management-api.md +++ b/docs-site/src/content/docs/reference/management-api.md @@ -552,6 +552,37 @@ All four rates must be finite numbers from 0 through 1,000,000, in USD per 1M to Unknown fields and malformed rates are rejected. A write preserves other models' overrides and returns `{ ok: true, provider, modelId, cost }`; reset returns `cost: null`. +### Automatic prices + +A model with no manual price is priced, in this order, from a verified official price, then the +public models.dev list, then the public OpenRouter list. The two public lists are fetched in the +background (at most daily, plus one fetch when a model no source could price is first seen), never +on a request path, and cached in `price-catalog-cache.json` in the OpenCodex home. Offline, the +previous snapshot keeps answering. A price from a public list is always an estimate. A manual price +always wins and is never scaled. + +Each non-combo row of `GET /api/models` carries `price` when a price is known and omits it when not: +`{ source, input, output, cacheRead, cacheWrite }` in USD per 1M tokens, where `source` is `manual`, +`official`, `models-dev` or `openrouter`. A public-list price also carries `fetchedAt` (ISO time). +`multiplier` appears when the provider's relay multiplier was applied, and `apiEquivalent: true` when +the provider bills through a subscription, so the figure is what the same tokens would cost on the API. + +`POST /api/pricing/refresh` forces an immediate refresh of both public lists, clears the retry +cooldowns for models that could not be priced, and returns +`{ ok: true, sources: { "models-dev": status, "openrouter": status } }`. A source `status` is +`{ ok: true, fetchedAt, count }` or `{ ok: false, error }`, where `error` is a short code such as +`fetch_failed`, `bad_json`, `empty` or `http_503`, never a response body. + +`PUT /api/providers/{provider}/price-multiplier` accepts `{ multiplier }`, a finite number greater +than 0 and at most 1000, or `null` to remove it, and returns `{ ok: true, provider, multiplier }`. +The multiplier models a relay that charges a markup or discount over the vendor's list price. It +scales official, models.dev and OpenRouter prices for that provider and never a manual price. +`GET /api/providers/{provider}/model-costs` also returns `priceMultiplier` when one is set. An unset +multiplier is absent from `config.json`; an invalid stored value is ignored at load with a warning. + +`/api/logs` and `/api/usage` cost results add the reasons `online_price_catalog`, +`relay_price_multiplier` and `subscription_api_equivalent` to `estimateReasons` when they apply. + The **Model price** dialog offers **Automatic**, **Flat rate**, and **Custom threshold** prompt-length pricing. Automatic (also the default when the policy is absent) retains catalog context rules with the saved base rates. Flat disables context adjustments. Custom selects four @@ -647,7 +678,21 @@ maps these facts come from. Display name is not part of this route; use `GET /api/models` reports the exact stored `contextWindowDeclared` separately from the effective `contextWindow`. The Models editor starts from the declaration and shows the effective -window as an inherited hint when no declaration exists. If a save outcome is unknown, or the +window as an inherited hint when no declaration exists. + +Routed rows also carry `capabilitySources`, one label per axis (`contextWindow`, +`inputModalities`, `reasoningEfforts`): `manual` (an operator declaration), `probed` (a +declaration written by the capability probe), `reported` (discovery, the registry or provider +metadata), `inferred` (a context window the catalog estimated under `inferContextWindow`), or +`unknown`. When the same model id is known elsewhere, `inferredCapabilities` carries the estimate +for each axis as `{ value, basis: { source, provider, modelId } }`, where `source` is `registry` +(a built-in original-vendor provider), `vendor` (bundled vendor metadata), `provider` (another +configured provider's declaration) or `aggregator` (bundled aggregator metadata), in that order of +authority. Ids match ignoring case, a `vendor/` prefix, dot-versus-hyphen spelling and a `[...]` +suffix; two sources of the same authority that disagree give no estimate. These estimates are +display data: they never route a request and never enter the Codex catalog on their own. To use +one, save it as a declaration (the model-settings route, `modelCapabilities`, or +`modelReasoningEfforts`) or, for the context window, enable `inferContextWindow`. If a save outcome is unknown, or the save succeeded but the model list did not reload, the dialog disables further writes and offers a read-only Reload. After a successful reload, reopen the row to inspect the stored values. A `catalogRefresh` that failed, or was skipped with `retryable: true`, leaves the save in place and @@ -770,6 +815,7 @@ already read. | `PATCH /api/providers?name=...` | Update allowed provider fields (including a merged `headers` block), enabled/default state, or OpenAI account mode | 400 invalid field or transition; 404 unknown provider | | `DELETE /api/providers?name=...` | Delete a provider, reassigning the default when possible | 404 unknown provider; 409 `last_provider`; 409 `provider_has_dependent_combos` | | `POST /api/providers/test?name=...` | Perform a bounded live provider connectivity/model-discovery probe | 404 unknown provider; failures are normally returned as `ok: false` evidence | +| `POST /api/providers/probe-capabilities` | Probe whether one model accepts an image input and which reasoning efforts it accepts, optionally writing the conclusive results back as labeled declarations (see [Capability probe](#capability-probe)) | 400 invalid request; 404 unknown provider; 409 probe already running for that provider | | `GET /api/provider-quotas` | Read provider quota reports; `refresh=1` forces refresh | — | | `GET /api/quota-pool` | Read the passive `ocx.quota-pool/1` account quota pool (per-account and pooled windows, opaque account references) from cached evidence only; never probes or refreshes, and ignores `refresh`. Pooled windows are a plan-weighted estimate over the routable accounts' readings from the last 30 minutes, plus an `availability` window while no account can run. Also readable from a Remote Link Child or Hub client at `/api/machine/hub-relay/api/quota-pool` | — | | `GET /api/quota-resets` | List recently detected quota-window resets and whether detection is enabled; `limit=` caps the count | 400 invalid `limit` | @@ -784,6 +830,56 @@ also preserves an omitted setting, including the latest stored `false`; an expli boolean wins. See [Context windows on custom providers](/reference/configuration/providers/#context-windows-on-custom-providers) for its default-off, custom-provider-only behavior. +#### Capability probe + +`POST /api/providers/probe-capabilities` sends a few tiny live requests to one model on one +configured provider. It reuses the explicit completion diagnostic (`openai-chat` and +`openai-responses` wires only, 32-token output cap, 20-second deadline per request, no retries), +changing only the request body. A plain prompt is sent first as a baseline; one probe adds a 1x1 PNG image beside the prompt, and one probe +per reasoning effort adds `reasoning_effort` (Chat) or `reasoning.effort` (Responses). + +Request body: + +```json +{ "name": "my-provider", "model": "vendor/model", "checks": ["image", "reasoning"], + "efforts": ["low", "medium", "high"], "confirmMetered": true, "apply": false } +``` + +- `checks` defaults to both. `efforts` is valid only with the `reasoning` check, accepts declared + effort names (`none`, `minimal`, `low` … `ultra`, at most 6), and defaults to the model's + configured ladder, else `low`, `medium`, `high`. +- A provider that authenticates with an API key and points at a non-local destination is + **metered**. Without `confirmMetered: true` nothing is sent and the response is + `{ "applicable": true, "sent": false, "requiresConfirmation": true, "metered": true, "plannedRequests": n }`. +- OAuth, forward-auth, disabled, and unsupported-adapter providers answer + `{ "applicable": false, "reason": ..., "sent": false }` without sending. + +When sent, the response is `{ applicable: true, sent: true, metered, provider, model, baseline, image?, reasoning?, applied? }`: + +- `baseline.status` reports the plain prompt. Unless it is `accepted`, no variant is sent and every + check is `inconclusive`, so an unrelated 400 (an unknown model, say) is never read as "this model + rejects images". `plannedRequests` counts the baseline. + +- `image.status` is `accepted` (the request completed), `rejected` (HTTP 400 or 422), or + `inconclusive` (timeout, network, 401/403/429/5xx, or an incomplete stream); `image.error` + carries only `{ code, status? }`. +- `reasoning` lists efforts under `accepted`, `rejected`, and `inconclusive`. A reasoning probe that + finishes without text still counts as accepted: the request was admitted and the small output cap + can be spent entirely on reasoning. +- Probes run one at a time. A 401, 403, or 429 (or a missing or changed credential) stops the run, + and the remaining efforts are reported as inconclusive. +- "Accepted" means the upstream did not refuse the field; a gateway that silently ignores + `reasoning_effort` reads as accepting every effort. Upstream response and error bodies are never + returned or logged. + +With `apply: true`, conclusive results are saved and the Codex catalog is refreshed, as after a +provider PATCH. An accepted image writes `modelCapabilities[model].inputModalities = ["text", "image"]`, +a rejected one `["text"]`; other capability axes are untouched. The reasoning ladder is written to +`modelReasoningEfforts[model]` (accepted efforts in ladder order) only when every probed effort was +conclusive. `applied` reports what was written. Each written axis is labeled in the provider's +`modelCapabilityProbes` field; a later manual edit of that axis through `PATCH /api/providers` +(`modelCapabilities`) or `PUT /api/model-settings` removes the label. + When Antigravity's quota-summary endpoint has no usable response, quota reports may also include per-model Cloud Code Assist usage windows when the model-discovery response provides them. These measurements are diagnostic; an absent model window does not establish that the model is unavailable. diff --git a/gui/src/components/ModelPriceDialog.tsx b/gui/src/components/ModelPriceDialog.tsx index 67f51b14690..beca7d3b042 100644 --- a/gui/src/components/ModelPriceDialog.tsx +++ b/gui/src/components/ModelPriceDialog.tsx @@ -5,6 +5,9 @@ import { readJsonOrThrow } from "../fetch-json"; import { useT, type TKey } from "../i18n/shared"; import type { ModelRow } from "../pages/models-shared"; import { Select } from "../ui"; +import { AutomaticPrice, RelayMultiplier } from "./ModelPriceExtras"; +import { parseMultiplierDraft, postPricingRefresh, putPriceMultiplier } from "./model-price-requests"; +import { adoptableCost } from "./model-price-source"; import { CUSTOM_FIELD, EMPTY_CUSTOM, EMPTY_RATES, INVALID, RATE_FIELDS, buildDraftCost, isRecord, loadDraft, parseModelCost, receiptMatches, type CustomDraft, type InvalidField, type ModelCost, type PricingMode, type RateDraft, type RateField, @@ -17,6 +20,7 @@ const RATE_LABELS: Record = { cacheWrite: "pricing.override.cacheWrite", }; const REQUEST_TIMEOUT_MS = 60_000; +type AuxKind = "refreshPrices" | "multiplier"; type Phase = "loading" | "loadFailed" | "ready" | "saving" | "unknown" | "refreshing" | "refreshFailed"; interface ModelPriceDialogProps { @@ -35,6 +39,7 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: const inputRef = useRef(null); const customFieldRefs = useRef>>({}); const submitRef = useRef(null); + const multiplierRef = useRef(null); const requestRef = useRef(null); const mutationPendingRef = useRef(false); const [phase, setPhase] = useState("loading"); @@ -45,6 +50,9 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: const [errorKey, setErrorKey] = useState(null); const [invalidField, setInvalidField] = useState(null); const [recovered, setRecovered] = useState(false); + const [multiplier, setMultiplier] = useState(null); + const [multiplierDraft, setMultiplierDraft] = useState(""); + const [aux, setAux] = useState(null); const endpoint = `${apiBase}/api/providers/${encodeURIComponent(model.provider)}/model-costs`; const mutating = phase === "saving" || phase === "refreshing"; const locked = phase !== "ready"; @@ -68,6 +76,10 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: setPricingMode(loaded.mode); setCustomDraft(loaded.custom); setHasOverride(cost !== undefined); + const rawMultiplier = result.priceMultiplier; + const loadedMultiplier = typeof rawMultiplier === "number" && Number.isFinite(rawMultiplier) && rawMultiplier > 0 ? rawMultiplier : null; + setMultiplier(loadedMultiplier); + setMultiplierDraft(loadedMultiplier === null ? "" : String(loadedMultiplier)); setErrorKey(null); setInvalidField(null); // This read recovers an editable snapshot, not ordering against an earlier @@ -82,7 +94,7 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: bounded.clear(); if (requestRef.current === bounded) requestRef.current = null; }); - }, [endpoint, model.id, model.provider, setCustomDraft, setErrorKey, setInvalidField, setPricingMode]); + }, [endpoint, model.id, model.provider, setCustomDraft, setErrorKey, setInvalidField, setMultiplier, setMultiplierDraft, setPricingMode]); useEffect(() => { const dialog = dialogRef.current; @@ -144,6 +156,56 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: } }; + // A server-confirmed change that is not a model-cost write: refresh the list, then stay open. + const runAux = async (kind: AuxKind, failKey: TKey, op: (signal: AbortSignal) => Promise) => { + if (requestRef.current || phase !== "ready") return; + const bounded = createBoundedFetch(REQUEST_TIMEOUT_MS); + requestRef.current = bounded; + setPhase("saving"); + setAux(kind); + mutationPendingRef.current = true; + setErrorKey(null); + setInvalidField(null); + let confirmed = false; + try { + await op(bounded.signal); + bounded.signal.throwIfAborted(); + if (requestRef.current !== bounded) return; + confirmed = true; + setPhase("refreshing"); + if (!await onRefresh(bounded.signal)) throw new Error("catalog refresh failed"); + bounded.signal.throwIfAborted(); + if (requestRef.current === bounded) setPhase("ready"); + } catch { + if (requestRef.current !== bounded) return; + setPhase(confirmed ? "refreshFailed" : "ready"); + setErrorKey(confirmed ? "pricing.override.refreshFailed" : failKey); + } finally { + bounded.clear(); + if (requestRef.current === bounded) { + requestRef.current = null; + mutationPendingRef.current = false; + setAux(null); + } + } + }; + + const refreshPrices = () => void runAux("refreshPrices", "pricing.auto.refreshFailed", signal => postPricingRefresh(apiBase, signal)); + + const saveMultiplier = () => { + const value = parseMultiplierDraft(multiplierDraft); + if (value === "invalid" || multiplierRef.current?.validity.badInput) { + setErrorKey("pricing.multiplier.invalid"); + multiplierRef.current?.focus(); + return; + } + void runAux("multiplier", "pricing.multiplier.failed", async signal => { + await putPriceMultiplier(apiBase, model.provider, value, signal); + setMultiplier(value); + }); + }; + const multiplierDirty = multiplierDraft.trim() !== (multiplier === null ? "" : String(multiplier)); + const requestClose = () => { if (!mutationPendingRef.current) onClose(); }; @@ -193,6 +255,9 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: {model.namespaced}
+ { const adopted = adoptableCost(model.price); if (adopted) void save(adopted); }} + onRefreshPrices={refreshPrices} />

{t("pricing.override.help")}

{t("pricing.override.promptHelp")}

{phase === "loading" &&

{t("pricing.override.loading")}

} @@ -266,6 +331,9 @@ export default function ModelPriceDialog({ model, apiBase, onRefresh, onClose }: ))}
} + { setMultiplierDraft(value); setErrorKey(null); }} onSave={saveMultiplier} /> {recovered &&

{t("pricing.override.recovered")}

} {errorKey && } diff --git a/gui/src/components/ModelPriceExtras.tsx b/gui/src/components/ModelPriceExtras.tsx new file mode 100644 index 00000000000..ac8a3fa9a3a --- /dev/null +++ b/gui/src/components/ModelPriceExtras.tsx @@ -0,0 +1,77 @@ +import type { KeyboardEvent, RefObject } from "react"; +import { useT } from "../i18n/shared"; +import type { ModelRowPrice } from "../pages/models-shared"; +import { PRICE_SOURCE_LABEL, adoptableCost, describePrice } from "./model-price-source"; + +interface AutomaticPriceProps { + price?: ModelRowPrice; + locked: boolean; + refreshing: boolean; + onAdopt: () => void; + onRefreshPrices: () => void; +} + +/** The row's current automatic price, with adopt-as-manual and a forced catalog refresh. */ +export function AutomaticPrice({ price, locked, refreshing, onAdopt, onRefreshPrices }: AutomaticPriceProps) { + const t = useT(); + const manual = price?.source === "manual"; + return ( +
+

{t("pricing.auto.title")}

+ {manual &&

{t("pricing.auto.manualInEffect")}

} + {!price &&

{t("pricing.auto.none")}

} + {price && !manual && <> +

{t(PRICE_SOURCE_LABEL[price.source])}

+ {describePrice(t, price).slice(1).map(line =>

{line}

)} + } +
+ {price && !manual && } + +
+ {price && !manual &&

{t("pricing.auto.adoptHelp")}

} +
+ ); +} + +interface RelayMultiplierProps { + id: string; + provider: string; + draft: string; + dirty: boolean; + invalid: boolean; + locked: boolean; + saving: boolean; + inputRef: RefObject; + onDraft: (value: string) => void; + onSave: () => void; +} + +export function RelayMultiplier({ id, provider, draft, dirty, invalid, locked, saving, inputRef, onDraft, onSave }: RelayMultiplierProps) { + const t = useT(); + const onKeyDown = (event: KeyboardEvent) => { + if (event.key !== "Enter") return; + event.preventDefault(); + if (dirty && !locked) onSave(); + }; + return ( +
+

{t("pricing.multiplier.title", { provider })}

+

{t("pricing.multiplier.help")}

+
+
+ + onDraft(event.target.value)} /> +
+ +
+
+ ); +} diff --git a/gui/src/components/PriceSourceChip.tsx b/gui/src/components/PriceSourceChip.tsx new file mode 100644 index 00000000000..154c4598c2c --- /dev/null +++ b/gui/src/components/PriceSourceChip.tsx @@ -0,0 +1,21 @@ +import { useT } from "../i18n/shared"; +import type { ModelRowPrice } from "../pages/models-shared"; +import { PRICE_SOURCE_LABEL, describePrice, type PriceSourceId } from "./model-price-source"; + +interface PriceSourceChipProps { + price?: ModelRowPrice; + /** Legacy flag on rows from a server that predates `price`. */ + manualPricing?: boolean; +} + +export default function PriceSourceChip({ price, manualPricing }: PriceSourceChipProps) { + const t = useT(); + const source: PriceSourceId = price?.source ?? (manualPricing === true ? "manual" : "unknown"); + const title = price ? describePrice(t, price).join("\n") : t(source === "manual" ? "pricing.override.badge" : "pricing.source.unknownHint"); + return ( + + {t(PRICE_SOURCE_LABEL[source])} + + ); +} diff --git a/gui/src/components/model-price-dialog.css b/gui/src/components/model-price-dialog.css index 95d39941ac1..20abfb10628 100644 --- a/gui/src/components/model-price-dialog.css +++ b/gui/src/components/model-price-dialog.css @@ -50,6 +50,11 @@ .model-price-band-title { font-weight: var(--weight-medium); margin-bottom: 2px; } .model-price-custom-controls { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); gap: var(--space-2); } .model-price-custom-controls .field-label { min-height: 2.5em; } +.model-price-auto { margin-bottom: 12px; padding-bottom: 12px; border-bottom: 1px solid var(--border); } +.model-price-auto > p { margin: 2px 0; } +.model-price-auto-actions { display: flex; flex-wrap: wrap; gap: var(--space-2); margin: 8px 0 4px; } +.model-price-multiplier-row { display: flex; align-items: flex-end; gap: var(--space-2); } +.model-price-multiplier-row .model-price-rate-field { flex: 1 1 auto; } .model-price-dialog .model-display-name-error { margin-top: 8px; } @media (max-width: 760px) { .model-price-mode-option { min-height: 44px; } diff --git a/gui/src/components/model-price-requests.ts b/gui/src/components/model-price-requests.ts new file mode 100644 index 00000000000..7bb926e4d72 --- /dev/null +++ b/gui/src/components/model-price-requests.ts @@ -0,0 +1,33 @@ +// Requests behind the price dialog's catalog refresh and relay-multiplier controls. +import { readJsonOrThrow } from "../fetch-json"; +import { isRecord } from "./model-price-cost"; + +const MAX_MULTIPLIER = 1000; + +/** Forces a refresh of the online price catalogs; throws when the receipt is invalid or every source failed. */ +export async function postPricingRefresh(apiBase: string, signal: AbortSignal): Promise { + const result = await readJsonOrThrow(await fetch(`${apiBase}/api/pricing/refresh`, { method: "POST", signal })); + signal.throwIfAborted(); + if (!isRecord(result) || result.ok !== true) throw new Error("invalid pricing refresh receipt"); + const statuses = isRecord(result.sources) ? Object.values(result.sources) : []; + if (statuses.length > 0 && statuses.every(status => isRecord(status) && status.ok === false)) throw new Error("price catalogs unavailable"); +} + +/** Saves (or with null clears) a provider's relay multiplier; throws unless the receipt echoes it. */ +export async function putPriceMultiplier(apiBase: string, provider: string, multiplier: number | null, signal: AbortSignal): Promise { + const response = await fetch(`${apiBase}/api/providers/${encodeURIComponent(provider)}/price-multiplier`, { + method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ multiplier }), signal, + }); + const result = await readJsonOrThrow(response); + signal.throwIfAborted(); + if (!isRecord(result) || result.ok !== true || result.provider !== provider || result.multiplier !== multiplier) { + throw new Error("invalid price-multiplier receipt"); + } +} + +export function parseMultiplierDraft(draft: string): number | null | "invalid" { + const text = draft.trim(); + if (text === "") return null; + const value = Number(text); + return Number.isFinite(value) && value > 0 && value <= MAX_MULTIPLIER ? value : "invalid"; +} diff --git a/gui/src/components/model-price-source.ts b/gui/src/components/model-price-source.ts new file mode 100644 index 00000000000..56dc55ee77b --- /dev/null +++ b/gui/src/components/model-price-source.ts @@ -0,0 +1,42 @@ +// Shared labels and formatting for where a model's price came from. +import type { TFn, TKey } from "../i18n/shared"; +import type { ModelRowPrice } from "../pages/models-shared"; +import { MAX_RATE, RATE_FIELDS, type ModelCost } from "./model-price-cost"; + +export type PriceSourceId = ModelRowPrice["source"] | "unknown"; + +export const PRICE_SOURCE_LABEL: Record = { + manual: "pricing.override.badge", + official: "pricing.source.official", + "models-dev": "pricing.source.models-dev", + openrouter: "pricing.source.openrouter", + unknown: "pricing.source.unknown", +}; + +const rateFormat = new Intl.NumberFormat(undefined, { maximumFractionDigits: 6 }); +export const formatRate = (value: number): string => rateFormat.format(value); + +export function ratesLine(t: TFn, price: ModelRowPrice): string { + return t("pricing.source.rates", { + input: formatRate(price.input), output: formatRate(price.output), + cacheRead: formatRate(price.cacheRead), cacheWrite: formatRate(price.cacheWrite), + }); +} + +/** Tooltip lines: source, rates, fetched time, relay multiplier, API-equivalent flag. */ +export function describePrice(t: TFn, price: ModelRowPrice): string[] { + const lines = [t(PRICE_SOURCE_LABEL[price.source]), ratesLine(t, price)]; + const fetched = price.fetchedAt ? new Date(price.fetchedAt) : null; + if (fetched && !Number.isNaN(fetched.getTime())) lines.push(t("pricing.source.fetched", { time: fetched.toLocaleString() })); + if (price.multiplier !== undefined) lines.push(t("pricing.source.multiplier", { multiplier: formatRate(price.multiplier) })); + if (price.apiEquivalent) lines.push(t("pricing.source.apiEquivalent")); + return lines; +} + +/** The automatic rates as a savable manual price; null when the price is already manual or not savable. */ +export function adoptableCost(price: ModelRowPrice | undefined): ModelCost | null { + if (!price || price.source === "manual") return null; + const round = (value: number) => Math.round(value * 1e6) / 1e6; + const cost = { input: round(price.input), output: round(price.output), cacheRead: round(price.cacheRead), cacheWrite: round(price.cacheWrite) }; + return RATE_FIELDS.every(field => Number.isFinite(cost[field]) && cost[field] >= 0 && cost[field] <= MAX_RATE) ? cost : null; +} diff --git a/gui/src/i18n/de.ts b/gui/src/i18n/de.ts index 234541ce8ca..51c5665e673 100644 --- a/gui/src/i18n/de.ts +++ b/gui/src/i18n/de.ts @@ -1301,6 +1301,8 @@ export const de: Record = { "logs.detail.source.jawcode": "jawcode-Katalog", "logs.detail.source.expected": "Expected-Preis-Overlay", "logs.detail.source.user": "Anbieter-konfiguriertes Preis-Overlay", + "logs.detail.source.models-dev": "models.dev-Preiskatalog", + "logs.detail.source.openrouter": "OpenRouter-Preiskatalog", "logs.detail.verification.verified": "Verifiziert", "logs.detail.verification.derived": "Vom Basismodell abgeleitet", "logs.detail.attempt.target": "Anbieter / Modell", @@ -1375,6 +1377,9 @@ export const de: Record = { "logs.detail.estimate.expected_price_overlay": "Ein verifizierter Expected-Listenpreis wurde verwendet.", "logs.detail.estimate.provider_cost_overlay": "Ein vom Anbieter konfiguriertes Preis-Overlay wurde verwendet.", "logs.detail.estimate.priority_lower_bound": "Der bestätigte Priority-Preis ist nicht verfügbar; die angezeigte Schätzung ist eine bekannte Untergrenze.", + "logs.detail.estimate.online_price_catalog": "Ein Preis aus einem Online-Katalog (models.dev oder OpenRouter) wurde verwendet.", + "logs.detail.estimate.relay_price_multiplier": "Der Relay-Multiplikator des Anbieters wurde auf den Preis angewendet.", + "logs.detail.estimate.subscription_api_equivalent": "Dieser Anbieter ist ein Abonnement; die Kosten sind eine API-äquivalente Schätzung und keine Abbuchung.", "logs.col.error": "Fehler", "logs.col.upstreamReason": "Upstream-Grund", "logs.col.failureDetail": "Fehlerdetail", @@ -4063,6 +4068,31 @@ export const de: Record = { "pricing.override.refresh": "Liste aktualisieren", "pricing.override.cancel": "Abbrechen", "pricing.override.close": "Schließen", + "pricing.source.official": "Offizieller Preis", + "pricing.source.models-dev": "models.dev-Preis", + "pricing.source.openrouter": "OpenRouter-Preis", + "pricing.source.unknown": "Preis unbekannt", + "pricing.source.unknownHint": "Für dieses Modell ist kein Preis bekannt; die Kosten werden daher als unbekannt angezeigt.", + "pricing.source.rates": "Pro 1 Mio. Token (USD): Eingabe {input}, Ausgabe {output}, Cache-Lesen {cacheRead}, Cache-Schreiben {cacheWrite}", + "pricing.source.fetched": "Abgerufen {time}", + "pricing.source.multiplier": "Relay-Multiplikator ×{multiplier}", + "pricing.source.apiEquivalent": "API-äquivalente Schätzung", + "pricing.auto.title": "Automatischer Preis", + "pricing.auto.none": "Für dieses Modell ist kein automatischer Preis bekannt.", + "pricing.auto.manualInEffect": "Für dieses Modell gilt ein manueller Preis; automatische Preise werden ignoriert.", + "pricing.auto.adopt": "Als manuell übernehmen", + "pricing.auto.adoptHelp": "Speichert diese Sätze als manuellen Preis dieses Modells. Manuelle Preise haben immer Vorrang und werden nie multipliziert.", + "pricing.auto.refresh": "Preise aktualisieren", + "pricing.auto.refreshing": "Wird aktualisiert…", + "pricing.auto.refreshFailed": "Die Online-Preiskataloge konnten nicht aktualisiert werden. Versuche es später erneut.", + "pricing.multiplier.title": "Relay-Multiplikator für {provider}", + "pricing.multiplier.label": "Multiplikator", + "pricing.multiplier.placeholder": "Nicht angewendet", + "pricing.multiplier.help": "Optional, pro Anbieter. Multipliziert die automatischen offiziellen Preise dieses Anbieters, für Relays, die ein festes Vielfaches des Listenpreises berechnen. Wird nie auf manuelle Preise angewendet. Leer lassen, um ihn nicht anzuwenden.", + "pricing.multiplier.save": "Multiplikator speichern", + "pricing.multiplier.saving": "Wird gespeichert…", + "pricing.multiplier.invalid": "Gib eine Zahl größer als 0 und höchstens 1000 ein oder lass das Feld leer.", + "pricing.multiplier.failed": "Der Multiplikator konnte nicht gespeichert werden. Prüfe den Wert und versuche es erneut.", "usage.range.custom": "Eigener Zeitraum", "usage.range.start": "Beginn (Ortszeit)", "usage.range.end": "Ende (Ortszeit)", diff --git a/gui/src/i18n/en.ts b/gui/src/i18n/en.ts index 76381be2404..2e17a92b1c8 100644 --- a/gui/src/i18n/en.ts +++ b/gui/src/i18n/en.ts @@ -1359,6 +1359,8 @@ export const en = { "logs.detail.source.jawcode": "jawcode catalog", "logs.detail.source.expected": "Expected price overlay", "logs.detail.source.user": "Provider-configured price overlay", + "logs.detail.source.models-dev": "models.dev price catalog", + "logs.detail.source.openrouter": "OpenRouter price catalog", "logs.detail.verification.verified": "Verified", "logs.detail.verification.derived": "Derived from base model", "logs.detail.attempt.target": "Provider / model", @@ -1433,6 +1435,9 @@ export const en = { "logs.detail.estimate.expected_price_overlay": "A verified expected list price was used.", "logs.detail.estimate.provider_cost_overlay": "A provider-configured price overlay was used.", "logs.detail.estimate.priority_lower_bound": "The confirmed Priority price is unavailable; the displayed estimate is a known lower bound.", + "logs.detail.estimate.online_price_catalog": "A price from an online catalog (models.dev or OpenRouter) was used.", + "logs.detail.estimate.relay_price_multiplier": "The provider's relay multiplier was applied to the price.", + "logs.detail.estimate.subscription_api_equivalent": "This provider is a subscription, so the cost is an API-equivalent estimate, not a charge.", "logs.col.error": "Error", "logs.col.upstreamReason": "Upstream reason", "logs.col.failureDetail": "Failure detail", @@ -4097,6 +4102,31 @@ export const en = { "pricing.override.refresh": "Refresh list", "pricing.override.cancel": "Cancel", "pricing.override.close": "Close", + "pricing.source.official": "Official price", + "pricing.source.models-dev": "models.dev price", + "pricing.source.openrouter": "OpenRouter price", + "pricing.source.unknown": "Unknown price", + "pricing.source.unknownHint": "No price is known for this model, so its cost shows as unknown.", + "pricing.source.rates": "Per 1M tokens (USD): input {input}, output {output}, cache read {cacheRead}, cache write {cacheWrite}", + "pricing.source.fetched": "Fetched {time}", + "pricing.source.multiplier": "Relay multiplier ×{multiplier}", + "pricing.source.apiEquivalent": "API-equivalent estimate", + "pricing.auto.title": "Automatic price", + "pricing.auto.none": "No automatic price is known for this model.", + "pricing.auto.manualInEffect": "A manual price is in effect for this model; automatic prices are ignored.", + "pricing.auto.adopt": "Adopt as manual", + "pricing.auto.adoptHelp": "Saves these rates as this model's manual price. Manual prices always win and are never multiplied.", + "pricing.auto.refresh": "Refresh prices", + "pricing.auto.refreshing": "Refreshing…", + "pricing.auto.refreshFailed": "Could not refresh the online price catalogs. Try again later.", + "pricing.multiplier.title": "Relay multiplier for {provider}", + "pricing.multiplier.label": "Multiplier", + "pricing.multiplier.placeholder": "Not applied", + "pricing.multiplier.help": "Optional, per provider. Multiplies this provider's automatic official prices, for relays that charge a fixed multiple of list price. Never applied to manual prices. Leave empty to not apply.", + "pricing.multiplier.save": "Save multiplier", + "pricing.multiplier.saving": "Saving…", + "pricing.multiplier.invalid": "Enter a number greater than 0 and at most 1000, or leave empty.", + "pricing.multiplier.failed": "Could not save the multiplier. Check the value and try again.", "usage.range.custom": "Custom date range", "usage.range.start": "Start (local time)", "usage.range.end": "End (local time)", diff --git a/gui/src/i18n/fr.ts b/gui/src/i18n/fr.ts index f0314b0c16e..cd12e028f78 100644 --- a/gui/src/i18n/fr.ts +++ b/gui/src/i18n/fr.ts @@ -1340,6 +1340,8 @@ export const fr: Record = { "logs.detail.source.jawcode": "Catalogue jawcode", "logs.detail.source.expected": "Remplacement par le tarif attendu", "logs.detail.source.user": "Remplacement par le tarif configuré pour le fournisseur", + "logs.detail.source.models-dev": "Catalogue de prix models.dev", + "logs.detail.source.openrouter": "Catalogue de prix OpenRouter", "logs.detail.verification.verified": "Vérifié", "logs.detail.verification.derived": "Dérivé du modèle de base", "logs.detail.attempt.target": "Fournisseur / modèle", @@ -1414,6 +1416,9 @@ export const fr: Record = { "logs.detail.estimate.expected_price_overlay": "Un tarif catalogue attendu et vérifié a été utilisé.", "logs.detail.estimate.provider_cost_overlay": "Un remplacement de tarif configuré pour le fournisseur a été utilisé.", "logs.detail.estimate.priority_lower_bound": "Le tarif Priority confirmé n’est pas disponible ; l’estimation affichée est une borne inférieure connue.", + "logs.detail.estimate.online_price_catalog": "Un prix issu d’un catalogue en ligne (models.dev ou OpenRouter) a été utilisé.", + "logs.detail.estimate.relay_price_multiplier": "Le multiplicateur de relais du fournisseur a été appliqué au prix.", + "logs.detail.estimate.subscription_api_equivalent": "Ce fournisseur fonctionne par abonnement ; le coût est une estimation équivalente API, pas un débit réel.", "logs.col.error": "Erreur", "logs.col.upstreamReason": "Motif en amont", "logs.col.failureDetail": "Détail de l'échec", @@ -4052,6 +4057,31 @@ export const fr: Record = { "pricing.override.refresh": "Actualiser la liste", "pricing.override.cancel": "Annuler", "pricing.override.close": "Fermer", + "pricing.source.official": "Prix officiel", + "pricing.source.models-dev": "Prix models.dev", + "pricing.source.openrouter": "Prix OpenRouter", + "pricing.source.unknown": "Prix inconnu", + "pricing.source.unknownHint": "Aucun prix n’est connu pour ce modèle ; son coût s’affiche donc comme inconnu.", + "pricing.source.rates": "Par million de tokens (USD) : entrée {input}, sortie {output}, lecture du cache {cacheRead}, écriture du cache {cacheWrite}", + "pricing.source.fetched": "Récupéré {time}", + "pricing.source.multiplier": "Multiplicateur du relais ×{multiplier}", + "pricing.source.apiEquivalent": "Estimation équivalente API", + "pricing.auto.title": "Prix automatique", + "pricing.auto.none": "Aucun prix automatique n’est connu pour ce modèle.", + "pricing.auto.manualInEffect": "Un prix manuel est en vigueur pour ce modèle ; les prix automatiques sont ignorés.", + "pricing.auto.adopt": "Adopter comme manuel", + "pricing.auto.adoptHelp": "Enregistre ces tarifs comme prix manuel de ce modèle. Les prix manuels l’emportent toujours et ne sont jamais multipliés.", + "pricing.auto.refresh": "Actualiser les prix", + "pricing.auto.refreshing": "Actualisation…", + "pricing.auto.refreshFailed": "Impossible d’actualiser les catalogues de prix en ligne. Réessayez plus tard.", + "pricing.multiplier.title": "Multiplicateur du relais pour {provider}", + "pricing.multiplier.label": "Multiplicateur", + "pricing.multiplier.placeholder": "Non appliqué", + "pricing.multiplier.help": "Facultatif, par fournisseur. Multiplie les prix officiels automatiques de ce fournisseur, pour les relais qui facturent un multiple fixe du tarif public. Jamais appliqué aux prix manuels. Laissez vide pour ne pas l’appliquer.", + "pricing.multiplier.save": "Enregistrer le multiplicateur", + "pricing.multiplier.saving": "Enregistrement…", + "pricing.multiplier.invalid": "Saisissez un nombre supérieur à 0 et au plus égal à 1000, ou laissez vide.", + "pricing.multiplier.failed": "Impossible d’enregistrer le multiplicateur. Vérifiez la valeur et réessayez.", "usage.range.custom": "Période personnalisée", "usage.range.start": "Début (heure locale)", "usage.range.end": "Fin (heure locale)", diff --git a/gui/src/i18n/ja.ts b/gui/src/i18n/ja.ts index b71b131dd49..515b4f16d2c 100644 --- a/gui/src/i18n/ja.ts +++ b/gui/src/i18n/ja.ts @@ -1242,6 +1242,8 @@ export const ja: Record = { "logs.detail.source.jawcode": "jawcode カタログ", "logs.detail.source.expected": "予想価格オーバーレイ", "logs.detail.source.user": "プロバイダー設定の価格オーバーレイ", + "logs.detail.source.models-dev": "models.dev 価格カタログ", + "logs.detail.source.openrouter": "OpenRouter 価格カタログ", "logs.detail.verification.verified": "検証済み", "logs.detail.verification.derived": "ベースモデルから派生", "logs.detail.attempt.target": "プロバイダー / モデル", @@ -1316,6 +1318,9 @@ export const ja: Record = { "logs.detail.estimate.expected_price_overlay": "検証済みの予想定価が使用されました。", "logs.detail.estimate.provider_cost_overlay": "プロバイダー設定の価格オーバーレイが使用されました。", "logs.detail.estimate.priority_lower_bound": "確認済みの Priority 価格を利用できないため、表示される見積もりは既知の下限です。", + "logs.detail.estimate.online_price_catalog": "オンラインカタログ(models.dev または OpenRouter)の価格が使用されました。", + "logs.detail.estimate.relay_price_multiplier": "プロバイダーのリレー倍率がこの価格に適用されました。", + "logs.detail.estimate.subscription_api_equivalent": "このプロバイダーはサブスクリプションのため、コストは API 換算の見積もりであり、実際の請求ではありません。", "logs.col.error": "エラー", "logs.col.upstreamReason": "上流の理由", "logs.col.failureDetail": "失敗の詳細", @@ -4085,6 +4090,31 @@ export const ja: Record = { "pricing.override.refresh": "一覧を更新", "pricing.override.cancel": "キャンセル", "pricing.override.close": "閉じる", + "pricing.source.official": "公式価格", + "pricing.source.models-dev": "models.dev の価格", + "pricing.source.openrouter": "OpenRouter の価格", + "pricing.source.unknown": "価格不明", + "pricing.source.unknownHint": "このモデルの価格は不明なため、コストは不明と表示されます。", + "pricing.source.rates": "100万トークンあたり(USD): 入力 {input}、出力 {output}、キャッシュ読み取り {cacheRead}、キャッシュ書き込み {cacheWrite}", + "pricing.source.fetched": "取得日時 {time}", + "pricing.source.multiplier": "リレー倍率 ×{multiplier}", + "pricing.source.apiEquivalent": "API 換算の見積もり", + "pricing.auto.title": "自動価格", + "pricing.auto.none": "このモデルの自動価格は不明です。", + "pricing.auto.manualInEffect": "このモデルには手動価格が設定されており有効です。自動価格は無視されます。", + "pricing.auto.adopt": "手動価格として採用", + "pricing.auto.adoptHelp": "これらの料金をこのモデルの手動価格として保存します。手動価格は常に優先され、倍率は適用されません。", + "pricing.auto.refresh": "価格を更新", + "pricing.auto.refreshing": "更新中…", + "pricing.auto.refreshFailed": "オンラインの価格カタログを更新できませんでした。後でもう一度お試しください。", + "pricing.multiplier.title": "{provider} のリレー倍率", + "pricing.multiplier.label": "倍率", + "pricing.multiplier.placeholder": "適用しない", + "pricing.multiplier.help": "任意、プロバイダーごとの設定です。このプロバイダーの自動の公式価格に倍率を掛けます。定価の固定倍額を請求するリレー向けです。手動価格には適用されません。空欄の場合は適用しません。", + "pricing.multiplier.save": "倍率を保存", + "pricing.multiplier.saving": "保存中…", + "pricing.multiplier.invalid": "0 より大きく 1000 以下の数値を入力するか、空欄にしてください。", + "pricing.multiplier.failed": "倍率を保存できませんでした。値を確認してもう一度お試しください。", "usage.range.custom": "期間を指定", "usage.range.start": "開始(現地時間)", "usage.range.end": "終了(現地時間)", diff --git a/gui/src/i18n/ko.ts b/gui/src/i18n/ko.ts index 399a227c363..729147af601 100644 --- a/gui/src/i18n/ko.ts +++ b/gui/src/i18n/ko.ts @@ -1340,6 +1340,8 @@ export const ko: Record = { "logs.detail.source.jawcode": "jawcode 카탈로그", "logs.detail.source.expected": "expected 가격 오버레이", "logs.detail.source.user": "프로바이더 구성 가격 오버레이", + "logs.detail.source.models-dev": "models.dev 가격 카탈로그", + "logs.detail.source.openrouter": "OpenRouter 가격 카탈로그", "logs.detail.verification.verified": "검증됨", "logs.detail.verification.derived": "기반 모델 유도", "logs.detail.attempt.target": "프로바이더 / 모델", @@ -1414,6 +1416,9 @@ export const ko: Record = { "logs.detail.estimate.expected_price_overlay": "검증된 expected 정가를 사용했습니다.", "logs.detail.estimate.provider_cost_overlay": "프로바이더 구성 가격 오버레이를 사용했습니다.", "logs.detail.estimate.priority_lower_bound": "확인된 Priority 가격을 사용할 수 없어 표시된 추정치는 알려진 하한입니다.", + "logs.detail.estimate.online_price_catalog": "온라인 카탈로그(models.dev 또는 OpenRouter)의 가격을 사용했습니다.", + "logs.detail.estimate.relay_price_multiplier": "프로바이더의 릴레이 배수가 이 가격에 적용되었습니다.", + "logs.detail.estimate.subscription_api_equivalent": "이 프로바이더는 구독형이므로 비용은 실제 청구가 아니라 API 환산 추정치입니다.", "logs.col.error": "오류", "logs.col.upstreamReason": "업스트림 원인", "logs.col.failureDetail": "실패 상세", @@ -4085,6 +4090,31 @@ export const ko: Record = { "pricing.override.refresh": "목록 갱신", "pricing.override.cancel": "취소", "pricing.override.close": "닫기", + "pricing.source.official": "공식 가격", + "pricing.source.models-dev": "models.dev 가격", + "pricing.source.openrouter": "OpenRouter 가격", + "pricing.source.unknown": "가격 알 수 없음", + "pricing.source.unknownHint": "이 모델의 가격을 알 수 없어 비용이 알 수 없음으로 표시됩니다.", + "pricing.source.rates": "100만 토큰당(USD): 입력 {input}, 출력 {output}, 캐시 읽기 {cacheRead}, 캐시 쓰기 {cacheWrite}", + "pricing.source.fetched": "가져온 시각 {time}", + "pricing.source.multiplier": "릴레이 배수 ×{multiplier}", + "pricing.source.apiEquivalent": "API 환산 추정치", + "pricing.auto.title": "자동 가격", + "pricing.auto.none": "이 모델의 자동 가격을 알 수 없습니다.", + "pricing.auto.manualInEffect": "이 모델에는 수동 가격이 적용 중이며 자동 가격은 무시됩니다.", + "pricing.auto.adopt": "수동 가격으로 채택", + "pricing.auto.adoptHelp": "이 요율을 이 모델의 수동 가격으로 저장합니다. 수동 가격은 항상 우선하며 배수가 적용되지 않습니다.", + "pricing.auto.refresh": "가격 새로고침", + "pricing.auto.refreshing": "새로고침 중…", + "pricing.auto.refreshFailed": "온라인 가격 카탈로그를 새로고침하지 못했습니다. 나중에 다시 시도하세요.", + "pricing.multiplier.title": "{provider}의 릴레이 배수", + "pricing.multiplier.label": "배수", + "pricing.multiplier.placeholder": "적용 안 함", + "pricing.multiplier.help": "선택 사항이며 프로바이더별로 설정합니다. 이 프로바이더의 자동 공식 가격에 배수를 곱하며, 정가의 고정 배수를 청구하는 릴레이용입니다. 수동 가격에는 적용되지 않습니다. 비워 두면 적용하지 않습니다.", + "pricing.multiplier.save": "배수 저장", + "pricing.multiplier.saving": "저장 중…", + "pricing.multiplier.invalid": "0보다 크고 1000 이하인 숫자를 입력하거나 비워 두세요.", + "pricing.multiplier.failed": "배수를 저장하지 못했습니다. 값을 확인하고 다시 시도하세요.", "usage.range.custom": "기간 직접 지정", "usage.range.start": "시작 (현지 시간)", "usage.range.end": "종료 (현지 시간)", diff --git a/gui/src/i18n/pt.ts b/gui/src/i18n/pt.ts index 1b5c4960b76..06f27c7c296 100644 --- a/gui/src/i18n/pt.ts +++ b/gui/src/i18n/pt.ts @@ -1354,6 +1354,8 @@ export const pt: Record = { "logs.detail.source.jawcode": "catálogo jawcode", "logs.detail.source.expected": "Sobreposição de preço esperado", "logs.detail.source.user": "Sobreposição de preço configurada no provedor", + "logs.detail.source.models-dev": "Catálogo de preços do models.dev", + "logs.detail.source.openrouter": "Catálogo de preços do OpenRouter", "logs.detail.verification.verified": "Verificado", "logs.detail.verification.derived": "Derivado do modelo base", "logs.detail.attempt.target": "Provedor / modelo", @@ -1428,6 +1430,9 @@ export const pt: Record = { "logs.detail.estimate.expected_price_overlay": "Foi usado um preço de tabela esperado verificado.", "logs.detail.estimate.provider_cost_overlay": "Foi usada uma sobreposição de preço configurada no provedor.", "logs.detail.estimate.priority_lower_bound": "O preço Priority confirmado está indisponível; a estimativa exibida é um limite inferior conhecido.", + "logs.detail.estimate.online_price_catalog": "Foi usado um preço de um catálogo online (models.dev ou OpenRouter).", + "logs.detail.estimate.relay_price_multiplier": "O multiplicador de relay do provedor foi aplicado ao preço.", + "logs.detail.estimate.subscription_api_equivalent": "Este provedor é uma assinatura, então o custo é uma estimativa equivalente à API, não uma cobrança.", "logs.col.error": "Erro", "logs.col.upstreamReason": "Motivo do upstream", "logs.col.failureDetail": "Detalhe da falha", @@ -4058,6 +4063,31 @@ export const pt: Record = { "pricing.override.refresh": "Atualizar lista", "pricing.override.cancel": "Cancelar", "pricing.override.close": "Fechar", + "pricing.source.official": "Preço oficial", + "pricing.source.models-dev": "Preço do models.dev", + "pricing.source.openrouter": "Preço do OpenRouter", + "pricing.source.unknown": "Preço desconhecido", + "pricing.source.unknownHint": "Nenhum preço é conhecido para este modelo, então o custo aparece como desconhecido.", + "pricing.source.rates": "Por 1 milhão de tokens (USD): entrada {input}, saída {output}, leitura de cache {cacheRead}, gravação de cache {cacheWrite}", + "pricing.source.fetched": "Obtido em {time}", + "pricing.source.multiplier": "Multiplicador do relay ×{multiplier}", + "pricing.source.apiEquivalent": "Estimativa equivalente à API", + "pricing.auto.title": "Preço automático", + "pricing.auto.none": "Nenhum preço automático é conhecido para este modelo.", + "pricing.auto.manualInEffect": "Um preço manual está em vigor para este modelo; os preços automáticos são ignorados.", + "pricing.auto.adopt": "Adotar como manual", + "pricing.auto.adoptHelp": "Salva estas tarifas como o preço manual deste modelo. Preços manuais sempre prevalecem e nunca são multiplicados.", + "pricing.auto.refresh": "Atualizar preços", + "pricing.auto.refreshing": "Atualizando…", + "pricing.auto.refreshFailed": "Não foi possível atualizar os catálogos de preços online. Tente novamente mais tarde.", + "pricing.multiplier.title": "Multiplicador do relay para {provider}", + "pricing.multiplier.label": "Multiplicador", + "pricing.multiplier.placeholder": "Não aplicado", + "pricing.multiplier.help": "Opcional, por provedor. Multiplica os preços oficiais automáticos deste provedor, para relays que cobram um múltiplo fixo do preço de tabela. Nunca é aplicado a preços manuais. Deixe vazio para não aplicar.", + "pricing.multiplier.save": "Salvar multiplicador", + "pricing.multiplier.saving": "Salvando…", + "pricing.multiplier.invalid": "Informe um número maior que 0 e no máximo 1000, ou deixe vazio.", + "pricing.multiplier.failed": "Não foi possível salvar o multiplicador. Verifique o valor e tente novamente.", "usage.range.custom": "Intervalo de datas personalizado", "usage.range.start": "Início (hora local)", "usage.range.end": "Fim (hora local)", diff --git a/gui/src/i18n/ru.ts b/gui/src/i18n/ru.ts index 2771adadf00..5bb4ce1173e 100644 --- a/gui/src/i18n/ru.ts +++ b/gui/src/i18n/ru.ts @@ -1318,6 +1318,8 @@ export const ru: Record = { "logs.detail.source.jawcode": "каталог jawcode", "logs.detail.source.expected": "Оверлей ожидаемых цен", "logs.detail.source.user": "Ценовой оверлей провайдера", + "logs.detail.source.models-dev": "Каталог цен models.dev", + "logs.detail.source.openrouter": "Каталог цен OpenRouter", "logs.detail.verification.verified": "Подтверждено", "logs.detail.verification.derived": "Выведено из базовой модели", "logs.detail.attempt.target": "Провайдер / модель", @@ -1392,6 +1394,9 @@ export const ru: Record = { "logs.detail.estimate.expected_price_overlay": "Использована подтверждённая ожидаемая цена из прайс-листа.", "logs.detail.estimate.provider_cost_overlay": "Использован ценовой оверлей провайдера.", "logs.detail.estimate.priority_lower_bound": "Подтверждённая цена Priority недоступна; показанная оценка является известной нижней границей.", + "logs.detail.estimate.online_price_catalog": "Использована цена из онлайн-каталога (models.dev или OpenRouter).", + "logs.detail.estimate.relay_price_multiplier": "К цене применён множитель ретранслятора провайдера.", + "logs.detail.estimate.subscription_api_equivalent": "Этот провайдер работает по подписке, поэтому стоимость — оценка в эквиваленте API, а не реальное списание.", "logs.col.error": "Ошибка", "logs.col.upstreamReason": "Причина от провайдера", "logs.col.failureDetail": "Подробности сбоя", @@ -4086,6 +4091,31 @@ export const ru: Record = { "pricing.override.refresh": "Обновить список", "pricing.override.cancel": "Отмена", "pricing.override.close": "Закрыть", + "pricing.source.official": "Официальная цена", + "pricing.source.models-dev": "Цена models.dev", + "pricing.source.openrouter": "Цена OpenRouter", + "pricing.source.unknown": "Цена неизвестна", + "pricing.source.unknownHint": "Цена этой модели неизвестна, поэтому стоимость отображается как неизвестная.", + "pricing.source.rates": "За 1 млн токенов (USD): вход {input}, выход {output}, чтение кэша {cacheRead}, запись кэша {cacheWrite}", + "pricing.source.fetched": "Получено {time}", + "pricing.source.multiplier": "Множитель ретранслятора ×{multiplier}", + "pricing.source.apiEquivalent": "Оценка в эквиваленте API", + "pricing.auto.title": "Автоматическая цена", + "pricing.auto.none": "Автоматическая цена этой модели неизвестна.", + "pricing.auto.manualInEffect": "Для этой модели действует ручная цена; автоматические цены игнорируются.", + "pricing.auto.adopt": "Принять как ручную", + "pricing.auto.adoptHelp": "Сохраняет эти ставки как ручную цену модели. Ручные цены всегда приоритетны и никогда не умножаются.", + "pricing.auto.refresh": "Обновить цены", + "pricing.auto.refreshing": "Обновление…", + "pricing.auto.refreshFailed": "Не удалось обновить онлайн-каталоги цен. Повторите попытку позже.", + "pricing.multiplier.title": "Множитель ретранслятора для {provider}", + "pricing.multiplier.label": "Множитель", + "pricing.multiplier.placeholder": "Не применяется", + "pricing.multiplier.help": "Необязательно, задаётся для каждого провайдера. Умножает автоматические официальные цены этого провайдера, для ретрансляторов, которые берут фиксированную кратность от прайс-листа. К ручным ценам не применяется. Оставьте пустым, чтобы не применять.", + "pricing.multiplier.save": "Сохранить множитель", + "pricing.multiplier.saving": "Сохранение…", + "pricing.multiplier.invalid": "Введите число больше 0 и не более 1000 или оставьте поле пустым.", + "pricing.multiplier.failed": "Не удалось сохранить множитель. Проверьте значение и повторите попытку.", "usage.range.custom": "Произвольный период", "usage.range.start": "Начало (местное время)", "usage.range.end": "Конец (местное время)", diff --git a/gui/src/i18n/tr.ts b/gui/src/i18n/tr.ts index c27997fc8b4..f6a211a85bb 100644 --- a/gui/src/i18n/tr.ts +++ b/gui/src/i18n/tr.ts @@ -1345,6 +1345,8 @@ export const tr: Record = { "logs.detail.source.jawcode": "katalog", "logs.detail.source.expected": "Beklenen fiyat", "logs.detail.source.user": "Kullanıcı tarafından yapılandırılan sağlayıcı fiyat katmanı", + "logs.detail.source.models-dev": "models.dev fiyat kataloğu", + "logs.detail.source.openrouter": "OpenRouter fiyat kataloğu", "logs.detail.verification.verified": "Doğrulandı", "logs.detail.verification.derived": "Taban modelden türetildi", "logs.detail.attempt.target": "Sağlayıcı / model", @@ -1419,6 +1421,9 @@ export const tr: Record = { "logs.detail.estimate.expected_price_overlay": "Doğrulanmış liste fiyatı kullanıldı.", "logs.detail.estimate.provider_cost_overlay": "Kullanıcı tarafından yapılandırılan bir sağlayıcı fiyat katmanı kullanıldı.", "logs.detail.estimate.priority_lower_bound": "Doğrulanan Priority fiyatı kullanılamıyor; gösterilen tahmin bilinen bir alt sınırdır.", + "logs.detail.estimate.online_price_catalog": "Çevrimiçi bir katalogdan (models.dev veya OpenRouter) fiyat kullanıldı.", + "logs.detail.estimate.relay_price_multiplier": "Sağlayıcının aktarıcı çarpanı fiyata uygulandı.", + "logs.detail.estimate.subscription_api_equivalent": "Bu sağlayıcı abonelik tabanlıdır; maliyet gerçek bir tahsilat değil, API eşdeğeri bir tahmindir.", "logs.col.error": "Hata", "logs.col.upstreamReason": "Yukarı akış nedeni", "logs.col.failureDetail": "Hata ayrıntısı", @@ -4086,6 +4091,31 @@ export const tr: Record = { "pricing.override.refresh": "Listeyi yenile", "pricing.override.cancel": "İptal", "pricing.override.close": "Kapat", + "pricing.source.official": "Resmi fiyat", + "pricing.source.models-dev": "models.dev fiyatı", + "pricing.source.openrouter": "OpenRouter fiyatı", + "pricing.source.unknown": "Fiyat bilinmiyor", + "pricing.source.unknownHint": "Bu model için bilinen bir fiyat yok; bu yüzden maliyet bilinmiyor olarak gösterilir.", + "pricing.source.rates": "1 milyon token başına (USD): girdi {input}, çıktı {output}, önbellek okuma {cacheRead}, önbellek yazma {cacheWrite}", + "pricing.source.fetched": "Alındığı zaman {time}", + "pricing.source.multiplier": "Aktarıcı çarpanı ×{multiplier}", + "pricing.source.apiEquivalent": "API eşdeğeri tahmin", + "pricing.auto.title": "Otomatik fiyat", + "pricing.auto.none": "Bu model için bilinen otomatik fiyat yok.", + "pricing.auto.manualInEffect": "Bu model için manuel fiyat geçerli; otomatik fiyatlar yok sayılır.", + "pricing.auto.adopt": "Manuel olarak benimse", + "pricing.auto.adoptHelp": "Bu oranları modelin manuel fiyatı olarak kaydeder. Manuel fiyatlar her zaman önceliklidir ve asla çarpılmaz.", + "pricing.auto.refresh": "Fiyatları yenile", + "pricing.auto.refreshing": "Yenileniyor…", + "pricing.auto.refreshFailed": "Çevrimiçi fiyat kataloğu yenilenemedi. Daha sonra tekrar deneyin.", + "pricing.multiplier.title": "{provider} için aktarıcı çarpanı", + "pricing.multiplier.label": "Çarpan", + "pricing.multiplier.placeholder": "Uygulanmıyor", + "pricing.multiplier.help": "İsteğe bağlı, sağlayıcı başına. Bu sağlayıcının otomatik resmi fiyatlarını çarpar; liste fiyatının sabit bir katını alan aktarıcılar içindir. Manuel fiyatlara asla uygulanmaz. Uygulamamak için boş bırakın.", + "pricing.multiplier.save": "Çarpanı kaydet", + "pricing.multiplier.saving": "Kaydediliyor…", + "pricing.multiplier.invalid": "0'dan büyük ve en çok 1000 olan bir sayı girin veya boş bırakın.", + "pricing.multiplier.failed": "Çarpan kaydedilemedi. Değeri kontrol edip tekrar deneyin.", "usage.range.custom": "Özel tarih aralığı", "usage.range.start": "Başlangıç (yerel saat)", "usage.range.end": "Bitiş (yerel saat)", diff --git a/gui/src/i18n/vi.ts b/gui/src/i18n/vi.ts index 930028388dc..68e5030be2d 100644 --- a/gui/src/i18n/vi.ts +++ b/gui/src/i18n/vi.ts @@ -1339,6 +1339,8 @@ export const vi: Record = { "logs.detail.source.jawcode": "Danh mục jawcode", "logs.detail.source.expected": "Giá niêm yết mong đợi (Expected price overlay)", "logs.detail.source.user": "Giá tuỳ chỉnh của provider (Provider-configured price overlay)", + "logs.detail.source.models-dev": "Danh mục giá models.dev", + "logs.detail.source.openrouter": "Danh mục giá OpenRouter", "logs.detail.verification.verified": "Đã xác minh (Verified)", "logs.detail.verification.derived": "Bắt nguồn từ model cơ sở (Derived from base model)", "logs.detail.attempt.target": "Nhà cung cấp / model", @@ -1413,6 +1415,9 @@ export const vi: Record = { "logs.detail.estimate.expected_price_overlay": "Một giá niêm yết kỳ vọng (expected list price) đã xác minh được sử dụng.", "logs.detail.estimate.provider_cost_overlay": "Sử dụng giá tuỳ chỉnh do provider cấu hình.", "logs.detail.estimate.priority_lower_bound": "Giá Ưu tiên (Priority price) đã xác nhận không khả dụng; ước tính hiển thị là giới hạn dưới đã biết.", + "logs.detail.estimate.online_price_catalog": "Đã dùng giá từ danh mục trực tuyến (models.dev hoặc OpenRouter).", + "logs.detail.estimate.relay_price_multiplier": "Hệ số relay của provider đã được áp dụng cho giá.", + "logs.detail.estimate.subscription_api_equivalent": "Provider này là gói đăng ký nên chi phí là ước tính tương đương API, không phải khoản bị trừ thực tế.", "logs.col.error": "Lỗi", "logs.col.upstreamReason": "Lý do upstream", "logs.col.failureDetail": "Chi tiết lỗi", @@ -4014,6 +4019,31 @@ export const vi: Record = { "pricing.override.refresh": "Làm mới danh sách", "pricing.override.cancel": "Hủy", "pricing.override.close": "Đóng", + "pricing.source.official": "Giá chính thức", + "pricing.source.models-dev": "Giá models.dev", + "pricing.source.openrouter": "Giá OpenRouter", + "pricing.source.unknown": "Giá không xác định", + "pricing.source.unknownHint": "Không biết giá của model này nên chi phí hiển thị là không xác định.", + "pricing.source.rates": "Mỗi 1 triệu token (USD): đầu vào {input}, đầu ra {output}, đọc cache {cacheRead}, ghi cache {cacheWrite}", + "pricing.source.fetched": "Lấy lúc {time}", + "pricing.source.multiplier": "Hệ số relay ×{multiplier}", + "pricing.source.apiEquivalent": "Ước tính tương đương API", + "pricing.auto.title": "Giá tự động", + "pricing.auto.none": "Không biết giá tự động của model này.", + "pricing.auto.manualInEffect": "Model này đang dùng giá thủ công; giá tự động bị bỏ qua.", + "pricing.auto.adopt": "Dùng làm giá thủ công", + "pricing.auto.adoptHelp": "Lưu các mức giá này làm giá thủ công của model. Giá thủ công luôn được ưu tiên và không bao giờ bị nhân hệ số.", + "pricing.auto.refresh": "Làm mới giá", + "pricing.auto.refreshing": "Đang làm mới…", + "pricing.auto.refreshFailed": "Không thể làm mới danh mục giá trực tuyến. Hãy thử lại sau.", + "pricing.multiplier.title": "Hệ số relay cho {provider}", + "pricing.multiplier.label": "Hệ số", + "pricing.multiplier.placeholder": "Không áp dụng", + "pricing.multiplier.help": "Tùy chọn, theo từng provider. Nhân giá chính thức tự động của provider này, dành cho relay tính phí gấp một bội số cố định của giá niêm yết. Không bao giờ áp dụng cho giá thủ công. Để trống nếu không áp dụng.", + "pricing.multiplier.save": "Lưu hệ số", + "pricing.multiplier.saving": "Đang lưu…", + "pricing.multiplier.invalid": "Nhập một số lớn hơn 0 và không quá 1000, hoặc để trống.", + "pricing.multiplier.failed": "Không thể lưu hệ số. Hãy kiểm tra giá trị và thử lại.", "usage.range.custom": "Phạm vi ngày tùy chỉnh", "usage.range.start": "Bắt đầu (giờ địa phương)", "usage.range.end": "Kết thúc (giờ địa phương)", diff --git a/gui/src/i18n/zh-TW.ts b/gui/src/i18n/zh-TW.ts index 33da090ccc0..fd81726b7f2 100644 --- a/gui/src/i18n/zh-TW.ts +++ b/gui/src/i18n/zh-TW.ts @@ -3267,6 +3267,8 @@ export const zhTW: Record = { "logs.detail.protocol.attempt": "嘗試 {ordinal}", "logs.detail.protocol.none": "此請求沒有記錄路徑資料。", "logs.detail.source.user": "供應商設定的價格覆蓋", + "logs.detail.source.models-dev": "models.dev 價格目錄", + "logs.detail.source.openrouter": "OpenRouter 價格目錄", "logs.detail.attempt.recovery.transient5xx": "暫時性 5xx", "logs.detail.attempt.recovery.connectionReset": "連線重設", "logs.detail.attempt.recovery.oauth401": "OAuth 重新驗證", @@ -3324,6 +3326,9 @@ export const zhTW: Record = { "logs.detail.resend.refusedFutile": "未重送:相同請求仍會失敗", "logs.detail.estimate.provider_cost_overlay": "已使用供應商設定的價格覆蓋。", "logs.detail.estimate.priority_lower_bound": "無法取得已確認的 Priority 價格;目前顯示的估算是已知下限。", + "logs.detail.estimate.online_price_catalog": "使用了線上價格目錄(models.dev 或 OpenRouter)中的價格。", + "logs.detail.estimate.relay_price_multiplier": "已對此價格套用供應商的中轉倍率。", + "logs.detail.estimate.subscription_api_equivalent": "此供應商為訂閱制,因此費用是依 API 價格折算的估算值,並非實際扣款。", "pws.cockpitImportDescription": "從此裝置匯入 Cockpit Tools Antigravity JSON 匯出檔。不會顯示檔案內容。", "pws.cockpitImportFileLabel": "Cockpit Tools Antigravity JSON 匯出檔", "pws.cockpitImportChooseFile": "選擇 JSON 檔案", @@ -4049,6 +4054,31 @@ export const zhTW: Record = { "pricing.override.refresh": "重新整理清單", "pricing.override.cancel": "取消", "pricing.override.close": "關閉", + "pricing.source.official": "官方價格", + "pricing.source.models-dev": "models.dev 價格", + "pricing.source.openrouter": "OpenRouter 價格", + "pricing.source.unknown": "價格未知", + "pricing.source.unknownHint": "此模型沒有已知價格,因此費用顯示為未知。", + "pricing.source.rates": "每百萬 token(美元):輸入 {input},輸出 {output},快取讀取 {cacheRead},快取寫入 {cacheWrite}", + "pricing.source.fetched": "取得於 {time}", + "pricing.source.multiplier": "中轉倍率 ×{multiplier}", + "pricing.source.apiEquivalent": "依 API 價格折算的估算值", + "pricing.auto.title": "自動價格", + "pricing.auto.none": "此模型沒有已知的自動價格。", + "pricing.auto.manualInEffect": "此模型已設定手動價格並正在生效,自動價格會被忽略。", + "pricing.auto.adopt": "採用為手動價格", + "pricing.auto.adoptHelp": "將這些費率儲存為此模型的手動價格。手動價格一律優先,且不會被倍率放大。", + "pricing.auto.refresh": "重新整理價格", + "pricing.auto.refreshing": "正在重新整理…", + "pricing.auto.refreshFailed": "無法重新整理線上價格目錄,請稍後再試。", + "pricing.multiplier.title": "{provider} 的中轉倍率", + "pricing.multiplier.label": "倍率", + "pricing.multiplier.placeholder": "不套用", + "pricing.multiplier.help": "選填,依供應商設定。會將此供應商的自動官方價格乘以此倍率,適用於按官方標價固定倍數收費的中轉站。不會套用到手動價格。留空表示不套用。", + "pricing.multiplier.save": "儲存倍率", + "pricing.multiplier.saving": "正在儲存…", + "pricing.multiplier.invalid": "請輸入大於 0 且不超過 1000 的數字,或留空。", + "pricing.multiplier.failed": "無法儲存倍率,請檢查數值後再試。", "usage.range.custom": "自訂時間範圍", "usage.range.start": "開始(本地時間)", "usage.range.end": "結束(本地時間)", diff --git a/gui/src/i18n/zh.ts b/gui/src/i18n/zh.ts index fe8f2d4c4d9..4541522f926 100644 --- a/gui/src/i18n/zh.ts +++ b/gui/src/i18n/zh.ts @@ -1313,6 +1313,8 @@ export const zh: Record = { "logs.detail.source.jawcode": "jawcode 目录", "logs.detail.source.expected": "Expected 价格覆盖", "logs.detail.source.user": "用户配置的提供方价格覆盖", + "logs.detail.source.models-dev": "models.dev 价格目录", + "logs.detail.source.openrouter": "OpenRouter 价格目录", "logs.detail.verification.verified": "已验证", "logs.detail.verification.derived": "由基础模型推导", "logs.detail.attempt.target": "提供方 / 模型", @@ -1387,6 +1389,9 @@ export const zh: Record = { "logs.detail.estimate.expected_price_overlay": "使用了已验证的 Expected 标价。", "logs.detail.estimate.provider_cost_overlay": "使用了用户配置的提供方价格覆盖。", "logs.detail.estimate.priority_lower_bound": "暂无已确认的 Priority 价格;当前显示的估算是已知下界。", + "logs.detail.estimate.online_price_catalog": "使用了在线价格目录(models.dev 或 OpenRouter)中的价格。", + "logs.detail.estimate.relay_price_multiplier": "已对该价格应用提供方的中转倍率。", + "logs.detail.estimate.subscription_api_equivalent": "该提供方是订阅制,因此费用是按 API 价格折算的估算值,并非实际扣费。", "logs.col.error": "错误", "logs.col.upstreamReason": "上游原因", "logs.col.failureDetail": "失败详情", @@ -4084,6 +4089,31 @@ export const zh: Record = { "pricing.override.refresh": "刷新列表", "pricing.override.cancel": "取消", "pricing.override.close": "关闭", + "pricing.source.official": "官方价格", + "pricing.source.models-dev": "models.dev 价格", + "pricing.source.openrouter": "OpenRouter 价格", + "pricing.source.unknown": "价格未知", + "pricing.source.unknownHint": "该模型没有已知价格,因此费用显示为未知。", + "pricing.source.rates": "每百万 token(美元):输入 {input},输出 {output},缓存读取 {cacheRead},缓存写入 {cacheWrite}", + "pricing.source.fetched": "获取于 {time}", + "pricing.source.multiplier": "中转倍率 ×{multiplier}", + "pricing.source.apiEquivalent": "按 API 价格折算的估算值", + "pricing.auto.title": "自动价格", + "pricing.auto.none": "该模型没有已知的自动价格。", + "pricing.auto.manualInEffect": "该模型已设置手动价格并正在生效,自动价格被忽略。", + "pricing.auto.adopt": "采用为手动价格", + "pricing.auto.adoptHelp": "把这些费率保存为该模型的手动价格。手动价格始终优先,且不会被倍率放大。", + "pricing.auto.refresh": "刷新价格", + "pricing.auto.refreshing": "正在刷新…", + "pricing.auto.refreshFailed": "无法刷新在线价格目录,请稍后重试。", + "pricing.multiplier.title": "{provider} 的中转倍率", + "pricing.multiplier.label": "倍率", + "pricing.multiplier.placeholder": "不应用", + "pricing.multiplier.help": "可选,按提供方设置。会把该提供方的自动官方价格乘以此倍率,适用于按官方标价固定倍数收费的中转站。不会应用到手动价格。留空表示不应用。", + "pricing.multiplier.save": "保存倍率", + "pricing.multiplier.saving": "正在保存…", + "pricing.multiplier.invalid": "请输入大于 0 且不超过 1000 的数字,或留空。", + "pricing.multiplier.failed": "无法保存倍率,请检查数值后重试。", "usage.range.custom": "自定义时间范围", "usage.range.start": "开始(本地时间)", "usage.range.end": "结束(本地时间)", diff --git a/gui/src/pages/Logs.tsx b/gui/src/pages/Logs.tsx index 9676f0ffbb8..94957c8d87d 100644 --- a/gui/src/pages/Logs.tsx +++ b/gui/src/pages/Logs.tsx @@ -83,7 +83,10 @@ type CostEstimateReason = | "cache_detail_missing" | "expected_price_overlay" | "provider_cost_overlay" - | "priority_lower_bound"; + | "priority_lower_bound" + | "online_price_catalog" + | "relay_price_multiplier" + | "subscription_api_equivalent"; type TokPerSecondResult = | { kind: "value"; value: number; estimated: boolean } @@ -97,7 +100,7 @@ interface MatchedPriceInfo { provider: string; modelId: string; jawcodeProvider?: string; - source: "jawcode" | "expected" | "user"; + source: "jawcode" | "expected" | "user" | "models-dev" | "openrouter"; sourceRef?: string; verifiedAt?: string; status: "verified" | "verified-derived"; @@ -342,6 +345,9 @@ const ESTIMATE_REASON_KEYS = { expected_price_overlay: "logs.detail.estimate.expected_price_overlay", provider_cost_overlay: "logs.detail.estimate.provider_cost_overlay", priority_lower_bound: "logs.detail.estimate.priority_lower_bound", + online_price_catalog: "logs.detail.estimate.online_price_catalog", + relay_price_multiplier: "logs.detail.estimate.relay_price_multiplier", + subscription_api_equivalent: "logs.detail.estimate.subscription_api_equivalent", } as const satisfies Record; /** diff --git a/gui/src/pages/Models.tsx b/gui/src/pages/Models.tsx index f7e093760e9..ea956aecc76 100644 --- a/gui/src/pages/Models.tsx +++ b/gui/src/pages/Models.tsx @@ -2,6 +2,7 @@ import { CodexStaleBanner } from "../components/codex-stale-banner"; import ModelCatalogSettingsPanels from "../components/ModelCatalogSettingsPanels"; import ModelDisplayNameDialog from "../components/ModelDisplayNameDialog"; import ModelPriceDialog from "../components/ModelPriceDialog"; +import PriceSourceChip from "../components/PriceSourceChip"; import ModelSettingsDialog from "../components/ModelSettingsDialog"; import { fetchCodexAppServerState, type AppServerStateOutcome } from "../codex-app-server-state"; import { ModelsReloadFocusBoundary } from "./models-reload-focus-boundary"; @@ -1728,7 +1729,7 @@ export default function Models({ apiBase, restartEpoch = 0, connected = false, c )} {aliases.models[provider]?.[m.id]?.source === "builtin" && {t("models.aliasAuto")}} {m.custom && {t("models.customBadge")}} - {!m.native && m.provider !== "combo" && m.manualPricing === true && {t("pricing.override.badge")}} + {m.provider !== "combo" && } {!m.custom && recentIds.has(m.id) && {t("models.newBadge")}} {m.contextCapped && {t("models.contextCappedValue", { value: fmtK(m.contextCap ?? contextCapValue) })}} @@ -2715,7 +2716,7 @@ export default function Models({ apiBase, restartEpoch = 0, connected = false, c {priceModel && ( row.namespaced === priceModel.namespaced) ?? priceModel} apiBase={apiBase} onRefresh={signal => load(true, signal)} onClose={() => { diff --git a/gui/src/pages/models-shared.ts b/gui/src/pages/models-shared.ts index ac54c6c73b7..a621aed4406 100644 --- a/gui/src/pages/models-shared.ts +++ b/gui/src/pages/models-shared.ts @@ -27,6 +27,18 @@ export function discoveryFailureLabel( } } +/** Effective per-1M-token price of a row and where it came from; absent on the row means unknown. */ +export interface ModelRowPrice { + source: "manual" | "official" | "models-dev" | "openrouter"; + input: number; + output: number; + cacheRead: number; + cacheWrite: number; + fetchedAt?: string; + multiplier?: number; + apiEquivalent?: true; +} + export interface ModelRow { provider: string; id: string; @@ -40,6 +52,7 @@ export interface ModelRow { displayNameOverride?: string; displayNameSource?: "operator" | "provider" | "fallback"; manualPricing?: boolean; + price?: ModelRowPrice; /** * Listed but currently unable to serve, because every usable target is quota-exhausted * (#1711). Distinct from `disabled`, which is the operator's own choice, and from visibility: diff --git a/gui/tests/models-price-editor.test.tsx b/gui/tests/models-price-editor.test.tsx index ef0a07c042c..1ff6ff6d341 100644 --- a/gui/tests/models-price-editor.test.tsx +++ b/gui/tests/models-price-editor.test.tsx @@ -40,6 +40,10 @@ describe("Models manual price editor", () => { let getGate: ReturnType | null; let putGate: ReturnType | null; let catalogGate: ReturnType | null; + let storedMultiplier: number | null; + let multiplierPuts: Array<{ multiplier: number | null }>; + let pricingRefreshes: number; + let pricingRefreshResponse: (() => Response) | null; let getResponse: (() => Response) | null; let putResponse: ((body: Mutation) => Response) | null; @@ -78,6 +82,10 @@ describe("Models manual price editor", () => { catalogGate = null; getResponse = null; putResponse = null; + storedMultiplier = null; + multiplierPuts = []; + pricingRefreshes = 0; + pricingRefreshResponse = null; testWindow.localStorage.setItem("ocx-lang", "en"); testWindow.localStorage.setItem("ocx-models-collapsed:v2", JSON.stringify([])); testWindow.sessionStorage.setItem("ocx.models.catalog.v1:http://localhost", JSON.stringify({ @@ -99,7 +107,21 @@ describe("Models manual price editor", () => { reads.push({ url, init }); if (getGate) await getGate.promise; if (getFailure) return Response.json({ error: "unavailable" }, { status: 503 }); - return getResponse ? getResponse() : Response.json({ provider: "xai-demo", modelCosts }); + return getResponse ? getResponse() : Response.json({ + provider: "xai-demo", modelCosts, ...(storedMultiplier === null ? {} : { priceMultiplier: storedMultiplier }), + }); + } + if (url.endsWith("/api/providers/xai-demo/price-multiplier")) { + const body = JSON.parse(String(init!.body)) as { multiplier: number | null }; + multiplierPuts.push(body); + storedMultiplier = body.multiplier; + return Response.json({ ok: true, provider: "xai-demo", multiplier: body.multiplier }); + } + if (url.endsWith("/api/pricing/refresh")) { + pricingRefreshes++; + return pricingRefreshResponse ? pricingRefreshResponse() : Response.json({ + ok: true, sources: { "models-dev": { ok: true, count: 3 }, openrouter: { ok: true, count: 2 } }, + }); } if (url.endsWith("/api/models")) { catalogReads++; @@ -153,7 +175,7 @@ describe("Models manual price editor", () => { } function inputs(): HTMLInputElement[] { - return [...container.querySelectorAll('dialog input[type="number"]')]; + return [...container.querySelectorAll('dialog input[type="number"]:not([id$="-multiplier"])')]; } function button(label: string): HTMLButtonElement { @@ -627,4 +649,99 @@ describe("Models manual price editor", () => { await flush(); expect(container.querySelector("dialog")).toBeNull(); }); + describe("price source, adoption, refresh and relay multiplier", () => { + const AUTO = { source: "models-dev" as const, input: 0.3, output: 1.2, cacheRead: 0.03, cacheWrite: 0, + fetchedAt: "2026-11-13T00:00:00.000Z" }; + + function chip(model: string): HTMLElement { + return trigger(model).closest(".models-model-row")!.querySelector("[data-price-source]")!; + } + + function setRows(price: ModelRow["price"]) { + rows = rows.map(row => row.id === "vendor/custom" ? { ...row, price } : row); + } + + test("every non-combo row shows its price source chip with rates in the tooltip", async () => { + rows = rows.map(row => row.id === "vendor/custom" + ? { ...row, price: { ...AUTO, multiplier: 1.5, apiEquivalent: true as const } } : row); + await mount(); + expect(chip("xai-demo/grok-4.6").dataset.priceSource).toBe("manual"); + const automatic = chip("xai-demo/vendor/custom"); + expect(automatic.dataset.priceSource).toBe("models-dev"); + expect(automatic.textContent).toBe("models.dev price"); + expect(automatic.title).toContain("input 0.3, output 1.2, cache read 0.03, cache write 0"); + expect(automatic.title).toContain("×1.5"); + expect(automatic.title).toContain("API-equivalent estimate"); + expect(container.querySelector("[aria-label='Edit price for combo/balanced']")).toBeNull(); + }); + + test("Automatic price block explains a manual price in effect and offers no adoption", async () => { + rows = rows.map(row => row.id === "grok-4.6" ? { ...row, price: { source: "manual" as const, ...SAVED } } : row); + await mount(); + await open(); + expect(container.querySelector("dialog")!.textContent).toContain("A manual price is in effect for this model"); + expect(button("Adopt as manual")).toBeUndefined(); + }); + + test("Adopt as manual saves the shown automatic rates through the model-costs PUT and closes", async () => { + setRows(AUTO); + await mount(); + await open("xai-demo/vendor/custom"); + expect(container.querySelector("dialog")!.textContent).toContain("models.dev price"); + await click("Adopt as manual"); + expect(mutations).toEqual([{ modelId: "vendor/custom", cost: { input: 0.3, output: 1.2, cacheRead: 0.03, cacheWrite: 0 } }]); + expect(container.querySelector("dialog")).toBeNull(); + }); + + test("Refresh prices posts, reloads the list, and keeps the dialog open showing the new price", async () => { + await mount(); + await open("xai-demo/vendor/custom"); + expect(container.querySelector("dialog")!.textContent).toContain("No automatic price is known"); + expect(chip("xai-demo/vendor/custom").dataset.priceSource).toBe("unknown"); + setRows(AUTO); + const before = catalogReads; + await click("Refresh prices"); + expect(pricingRefreshes).toBe(1); + expect(catalogReads).toBeGreaterThan(before); + expect(container.querySelector("dialog")).not.toBeNull(); + expect(container.querySelector("dialog")!.textContent).toContain("models.dev price"); + expect(button("Adopt as manual").disabled).toBe(false); + expect(chip("xai-demo/vendor/custom").dataset.priceSource).toBe("models-dev"); + }); + + test("Refresh prices reports failure when every source failed and unlocks editing", async () => { + pricingRefreshResponse = () => Response.json({ ok: true, sources: { "models-dev": { ok: false }, openrouter: { ok: false } } }); + await mount(); + await open("xai-demo/vendor/custom"); + await click("Refresh prices"); + expect(container.querySelector('dialog [role="alert"]')!.textContent).toContain("Could not refresh the online price catalogs"); + expect(button("Save").disabled).toBe(false); + }); + + test("relay multiplier loads from the GET, saves a number, clears with null, and rejects invalid text", async () => { + storedMultiplier = 2; + await mount(); + await open(); + const field = () => container.querySelector('dialog input[id$="-multiplier"]')!; + const setField = async (value: string) => act(async () => { + Object.getOwnPropertyDescriptor(testWindow.HTMLInputElement.prototype, "value")!.set!.call(field(), value); + field().dispatchEvent(new testWindow.Event("input", { bubbles: true })); + }); + expect(field().value).toBe("2"); + expect(button("Save multiplier").disabled).toBe(true); + await setField("1500"); + await click("Save multiplier"); + expect(multiplierPuts).toHaveLength(0); + expect(container.querySelector('dialog [role="alert"]')!.textContent).toContain("greater than 0 and at most 1000"); + await setField("1.25"); + await click("Save multiplier"); + expect(multiplierPuts).toEqual([{ multiplier: 1.25 }]); + expect(mutations).toHaveLength(0); + expect(container.querySelector("dialog")).not.toBeNull(); + await setField(""); + await click("Save multiplier"); + expect(multiplierPuts.at(-1)).toEqual({ multiplier: null }); + expect(field().value).toBe(""); + }); + }); }); diff --git a/scripts/test-layout/layout-providers.json b/scripts/test-layout/layout-providers.json index 681d70c87ba..4f493028d35 100644 --- a/scripts/test-layout/layout-providers.json +++ b/scripts/test-layout/layout-providers.json @@ -1,6 +1,9 @@ { "cursor-auto-tool-result-replay.test.ts": "providers/cursor", "provider-alias-template.test.ts": "providers", + "model-capability-inference.test.ts": "providers", + "provider-capability-probe.test.ts": "providers", + "provider-capability-probe-provenance.test.ts": "server", "provider-passive-wire-model.test.ts": "providers", "passive-key-observation.test.ts": "providers", "claude-chat-gateway-tool-schema.test.ts": "providers", diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 05605249cbe..519eb09dfa9 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -1244,6 +1244,7 @@ "model-cache-generation-tombstone.test.ts": "codex-integration", "model-cache.test.ts": "codex-integration", "model-costs-management-api.test.ts": "server", + "price-catalog-management-api.test.ts": "server", "model-discovery-management-api.test.ts": "server", "server-new-model-policy-arrival.test.ts": "server", "model-display-names-management-api.test.ts": "codex-integration", @@ -1864,7 +1865,7 @@ "usage-anthropic-fast-pricing.test.ts": "usage", "usage-cursor-fast-pricing.test.ts": "usage", "usage-attempt-delivery.test.ts": "usage", - "usage-cost.test.ts": "usage", "usage-haiku-5-5-pricing.test.ts": "usage", "usage-fable-5-2-pricing.test.ts": "usage", "gpt61-terra-pricing.test.ts": "usage", + "usage-cost.test.ts": "usage", "usage-haiku-5-5-pricing.test.ts": "usage", "usage-fable-5-2-pricing.test.ts": "usage", "gpt61-terra-pricing.test.ts": "usage", "price-catalog.test.ts": "usage", "price-catalog-miss.test.ts": "usage", "price-catalog-resolution.test.ts": "usage", "usage-debug.test.ts": "usage", "usage-failure-fingerprint.test.ts": "usage", "usage-failure-persistence.test.ts": "usage", diff --git a/src/config/load-degrade.ts b/src/config/load-degrade.ts index ec5a9684760..02b78b274b7 100644 --- a/src/config/load-degrade.ts +++ b/src/config/load-degrade.ts @@ -17,6 +17,7 @@ import { autoReviewModelOverridesConfigError, autoReviewModelTargetConfigError, modelCapabilitiesConfigError, + modelCapabilityProbesConfigError, providerInputModalitiesConfigError, sanitizeModelCapabilitiesForLoad, modelDisplayNamesConfigError, @@ -43,6 +44,7 @@ import { managementIngressSchema, codexPoolSchema, codexAccountAutoSwitchThresholdsSchema, + priceMultiplierConfigError, providerModelCostsConfigError, credentialGroupsSchema, hubConfigSchema, @@ -373,6 +375,11 @@ export function sanitizeCapabilityDeclarationsForLoad(parsed: unknown, warn: (me warn(`config.json provider ${JSON.stringify(redactSecretString(name))} has malformed inputModalities; restricting input modalities to text`); provider.inputModalities = ["text"]; } + // Probe provenance is a label, not a declaration: a malformed hand edit loses the label only. + if (modelCapabilityProbesConfigError(provider.modelCapabilityProbes) !== null) { + warn(`config.json provider ${JSON.stringify(redactSecretString(name))} has malformed modelCapabilityProbes; dropping probe provenance`); + delete provider.modelCapabilityProbes; + } if (provider.modelCapabilities === undefined) continue; if (modelCapabilitiesConfigError(provider.modelCapabilities) !== null) { warn(`config.json provider ${JSON.stringify(redactSecretString(name))} has malformed modelCapabilities; retaining valid axes and restricting malformed input modalities to text`); @@ -403,6 +410,11 @@ export function sanitizeModelCostsForLoad(parsed: unknown, warn: (message: strin const safeProviderName = JSON.stringify(redactSecretString(name)); if (!provider || typeof provider !== "object" || Array.isArray(provider)) continue; const p = provider as Record; + // A malformed relay multiplier is dropped with a warning rather than failing the config parse. + if (priceMultiplierConfigError(p.priceMultiplier) !== null) { + delete p.priceMultiplier; + warn(`⚠️ config.json providers.${safeProviderName}.priceMultiplier is invalid — ignoring the multiplier`); + } const costs = p.modelCosts; if (costs === undefined) continue; if (!costs || typeof costs !== "object" || Array.isArray(costs)) { diff --git a/src/config/provider-validation.ts b/src/config/provider-validation.ts index 49c9e202f13..a2a8068ad23 100644 --- a/src/config/provider-validation.ts +++ b/src/config/provider-validation.ts @@ -598,3 +598,94 @@ export function sanitizeModelCapabilitiesForLoad(value: unknown): Record MODEL_DISCOVERY_MAX_MODELS) return "modelCapabilityProbes has too many models"; + for (const [id, row] of Object.entries(value)) { + if (!isValidModelDiscoveryModelId(id) || ["__proto__", "prototype", "constructor"].includes(id)) { + return "modelCapabilityProbes keys must be exact non-reserved model ids"; + } + if (!capabilityRecord(row)) return "modelCapabilityProbes rows must be plain objects"; + if (Object.keys(row).some(key => key !== "probedAt" && !(PROBE_AXES as readonly string[]).includes(key))) { + return "modelCapabilityProbes contains an unknown field"; + } + if (typeof row.probedAt !== "string" || !PROBED_AT_PATTERN.test(row.probedAt) || Number.isNaN(Date.parse(row.probedAt))) { + return "modelCapabilityProbes probedAt must be an ISO 8601 UTC timestamp"; + } + for (const axis of PROBE_AXES) { + if (Object.hasOwn(row, axis) && row[axis] !== true) return `modelCapabilityProbes ${axis} must be true`; + } + if (!PROBE_AXES.some(axis => row[axis] === true)) return "modelCapabilityProbes rows must name at least one probed axis"; + } + return null; +} + +/** + * Rebuild `modelCapabilityProbes` so a flag never outlives the declaration it labels, optionally + * dropping named `[model, axis]` flags first (a manual edit wins). Always writes a fresh record and + * never mutates the stored one, because PATCH works on a shallow copy of the live row. The field is + * removed when nothing is left. Returns whether any flag was dropped. + */ +export function pruneModelCapabilityProbes( + provider: OcxProviderConfig, + dropped: Iterable = [], +): boolean { + const probes = provider.modelCapabilityProbes; + if (probes === undefined) return false; + const drop = new Set([...dropped].map(([id, axis]) => `${axis}\u0000${id}`)); + const declared = (id: string, axis: ModelCapabilityProbeAxis) => axis === "inputModalities" + ? Array.isArray(provider.modelCapabilities && Object.hasOwn(provider.modelCapabilities, id) ? provider.modelCapabilities[id]?.inputModalities : undefined) + : Array.isArray(provider.modelReasoningEfforts && Object.hasOwn(provider.modelReasoningEfforts, id) ? provider.modelReasoningEfforts[id] : undefined); + const next: NonNullable = {}; + let changed = false; + for (const [id, row] of Object.entries(probes)) { + if (["__proto__", "prototype", "constructor"].includes(id)) { changed = true; continue; } + const kept: NonNullable[string] = { probedAt: row.probedAt }; + for (const axis of PROBE_AXES) { + if (row[axis] !== true) continue; + if (!drop.has(`${axis}\u0000${id}`) && declared(id, axis)) kept[axis] = true; + else changed = true; + } + if (kept.inputModalities || kept.reasoningEfforts) next[id] = kept; + } + if (Object.keys(next).length > 0) provider.modelCapabilityProbes = next; + else delete provider.modelCapabilityProbes; + return changed; +} + +/** A PATCH of `modelCapabilities` that touches a model's input modalities makes that axis manual. */ +export function dropPatchedModelCapabilityProbes(provider: OcxProviderConfig, patch: unknown): void { + const probes = provider.modelCapabilityProbes; + if (probes === undefined) return; + const ids = patch === null ? Object.keys(probes) : capabilityRecord(patch) + ? Object.entries(patch).filter(([, row]) => row === null || (capabilityRecord(row) && Object.hasOwn(row, "inputModalities"))).map(([id]) => id) + : []; + pruneModelCapabilityProbes(provider, ids.map(id => [id, "inputModalities"] as const)); +} + +/** + * Same-name `POST /api/providers` overwrite: the provider form never sends the provenance, so an + * omitted field carries the live row's flags; then every flag whose declaration did not survive the + * overwrite is dropped. A submitted value is validated by `providerCompatFieldConfigError`. + */ +export function carryModelCapabilityProbes( + candidate: OcxProviderConfig, + submitted: Record, + live: OcxProviderConfig | undefined, +): void { + if (!Object.hasOwn(submitted, "modelCapabilityProbes") && live?.modelCapabilityProbes) { + candidate.modelCapabilityProbes = structuredClone(live.modelCapabilityProbes); + } + pruneModelCapabilityProbes(candidate); +} diff --git a/src/config/schema/config-schema.ts b/src/config/schema/config-schema.ts index ceeeafb6a7a..cd349e0a48c 100644 --- a/src/config/schema/config-schema.ts +++ b/src/config/schema/config-schema.ts @@ -46,6 +46,7 @@ import { memoryModelSettingSchema, memoryModelsSchema, modelPreferHostedToolsConfigError, + priceMultiplierConfigError, providerModelCostsConfigError, providerRelativeSendPathConfigError, } from "./leaf-validators"; @@ -655,6 +656,14 @@ export const configSchema = z.object({ message: modelCostsError, }); } + const priceMultiplierError = priceMultiplierConfigError((provider as { priceMultiplier?: unknown }).priceMultiplier); + if (priceMultiplierError) { + ctx.addIssue({ + code: "custom", + path: ["providers", redactSecretString(name), "priceMultiplier"], + message: priceMultiplierError, + }); + } const modelDisplayNamesError = modelDisplayNamesConfigError( (provider as { modelDisplayNames?: unknown }).modelDisplayNames, ); diff --git a/src/config/schema/leaf-validators.ts b/src/config/schema/leaf-validators.ts index 24984add74c..e5d4dbd2fcb 100644 --- a/src/config/schema/leaf-validators.ts +++ b/src/config/schema/leaf-validators.ts @@ -14,6 +14,7 @@ import { normalizeNonBlankStringArray, normalizeAutoReviewModelOverrides, modelCapabilitiesConfigError, + modelCapabilityProbesConfigError, providerInputModalitiesConfigError, contextTierRecordConfigError, mergeModelCapabilities, @@ -40,7 +41,7 @@ import { import { fastWireDeclarationError } from "../../providers/fastwire"; import { getProviderRegistryEntry, providerMatchesRegistryTransport, providerModelWireDefault } from "../../providers/registry"; import { resolveOpenAiVirtualModel } from "../../providers/openai-virtual-models"; -import { COST4_RATE_KEYS, isValidCost4Rate, isValidPromptLengthPricing, copyPromptLengthPricing } from "../../usage/user-cost-overlays"; +import { COST4_RATE_KEYS, isValidCost4Rate, isValidPriceMultiplier, isValidPromptLengthPricing, copyPromptLengthPricing } from "../../usage/user-cost-overlays"; import { MAX_COST4_RATE } from "../../usage/expected-prices"; import { DECLARABLE_HOSTED_TOOL_TYPES, @@ -330,6 +331,10 @@ export const providerConfigSchema = z.object({ // Raw diagnostics/writes reject malformed provenance; tolerant loads preserve the custom row. anthropicOAuthInstance: z.literal("anthropic2").optional().catch(undefined), modelCapabilities: modelCapabilitiesSchema.optional(), + modelCapabilityProbes: z.unknown().superRefine((value, ctx) => { + const error = modelCapabilityProbesConfigError(value); + if (error) ctx.addIssue({ code: "custom", message: error }); + }).optional().transform(value => value as OcxProviderConfig["modelCapabilityProbes"]), inputModalities: z.unknown().superRefine((value, ctx) => { const error = providerInputModalitiesConfigError(value); if (error) ctx.addIssue({ code: "custom", message: error }); @@ -517,6 +522,12 @@ export function providerModelCostsConfigError(value: unknown, field = "modelCost return null; } +/** Validate `providers..priceMultiplier`: absent, or a finite number in (0, 1000]. */ +export function priceMultiplierConfigError(value: unknown, field = "priceMultiplier"): string | null { + if (value === undefined) return null; + return isValidPriceMultiplier(value) ? null : `${field} must be a finite number greater than 0 and at most 1000`; +} + /** * Serialize `providers..modelCosts` for display: copy the four validated * rates and complete validated policy and DROP secret-shaped model ids, so a pasted diff --git a/src/providers/model-capability-inference.ts b/src/providers/model-capability-inference.ts new file mode 100644 index 00000000000..7850395fd79 --- /dev/null +++ b/src/providers/model-capability-inference.ts @@ -0,0 +1,154 @@ +import { listModelMetadata, resolveMetadataProvider, type ModelMetadata } from "../generated/model-metadata"; +import { modelRecordValue } from "../reasoning-effort"; +import type { OcxConfig } from "../types"; +import { PROVIDER_REGISTRY } from "./registry"; + +/** + * Where an inferred capability came from. `registry` is a built-in original-vendor provider entry, + * `vendor` / `aggregator` are bundled model metadata, `provider` is another configured provider's + * explicit declaration for the same model id. + */ +export interface CapabilityInferenceBasis { + source: "registry" | "vendor" | "provider" | "aggregator"; + provider: string; + modelId: string; +} + +export interface InferredCapability { + value: T; + basis: CapabilityInferenceBasis; +} + +export interface InferredModelCapabilities { + contextWindow?: InferredCapability; + inputModalities?: InferredCapability; + reasoningEfforts?: InferredCapability; +} + +type Axis = keyof InferredModelCapabilities; +type AxisValue = number | string[]; +interface Candidate { value: AxisValue; basis: CapabilityInferenceBasis } +type Tier = Map; + +// Original vendors only: a hosting vendor or reseller entry is not evidence of the model's own +// capability. Mirrors the vendor list of `src/codex/catalog/vendor-context.ts`. +const VENDORS = new Set(["anthropic", "openai", "google", "moonshot", "minimax", "deepseek", "xai", "zai", "mistral", "xiaomi"]); +const AGGREGATORS = ["openrouter", "opencode-go"]; +const MULTI_VENDOR_HOSTS = new Set(["google-vertex", "google-antigravity"]); + +/** Same-model key: case, a `vendor/` prefix, dot-vs-hyphen spelling and a `[...]` suffix are ignored. */ +export function capabilityInferenceKey(modelId: string): string { + const bare = modelId.trim().replace(/\[[^\]]*\]$/, ""); + return bare.slice(bare.lastIndexOf("/") + 1).toLowerCase().replaceAll(".", "-"); +} + +const sameValue = (a: AxisValue, b: AxisValue): boolean => JSON.stringify(a) === JSON.stringify(b); + +function addCandidate(tier: Tier, modelId: string, value: AxisValue | undefined, basis: Omit): void { + if (value === undefined || (Array.isArray(value) ? value.length === 0 : !(Number.isSafeInteger(value) && value > 0))) return; + const key = capabilityInferenceKey(modelId); + if (!key) return; + const existing = tier.get(key); + // Two sources in one tier that disagree answer nothing, and the lookup does not fall through to a + // lower tier either: a weaker source must not win just because a stronger one is ambiguous. + if (existing === undefined) tier.set(key, { value: Array.isArray(value) ? [...value] : value, basis: { ...basis, modelId } }); + else if (existing !== null && !sameValue(existing.value, value)) tier.set(key, null); +} + +function metadataTier(rows: readonly ModelMetadata[], source: "vendor" | "aggregator", axis: Axis): Tier { + const tier: Tier = new Map(); + for (const row of rows) { + if (row.id.startsWith("~")) continue; + const value = axis === "contextWindow" ? row.contextWindow : axis === "inputModalities" ? row.input : undefined; + addCandidate(tier, row.id, value, { source, provider: row.provider }); + } + return tier; +} + +function registryTier(axis: Axis): Tier { + const tier: Tier = new Map(); + for (const entry of PROVIDER_REGISTRY) { + if (!VENDORS.has(entry.id) && !VENDORS.has(resolveMetadataProvider(entry.id) ?? "")) continue; + // Multi-vendor hosts reachable through a vendor alias (Vertex, Antigravity) carry their own + // ladders and windows for other vendors' models; they are hosts, not the model's vendor. + if (MULTI_VENDOR_HOSTS.has(entry.id)) continue; + const record = axis === "contextWindow" ? entry.modelContextWindows + : axis === "inputModalities" ? entry.modelInputModalities : entry.modelReasoningEfforts; + for (const [modelId, value] of Object.entries(record ?? {})) { + addCandidate(tier, modelId, value as AxisValue, { source: "registry", provider: entry.id }); + } + } + return tier; +} + +function configuredTier(config: OcxConfig | undefined, exclude: string | undefined, axis: Axis): Tier { + const tier: Tier = new Map(); + for (const [name, provider] of Object.entries(config?.providers ?? {})) { + if (name === exclude || provider.disabled === true) continue; + const ids = new Set([ + ...Object.keys(provider.modelContextWindows ?? {}), + ...Object.keys(provider.modelCapabilities ?? {}), + ...Object.keys(provider.modelReasoningEfforts ?? {}), + ]); + for (const modelId of ids) { + const value = axis === "contextWindow" ? modelRecordValue(provider.modelContextWindows, modelId) + : axis === "inputModalities" ? provider.modelCapabilities?.[modelId]?.inputModalities + : modelRecordValue(provider.modelReasoningEfforts, modelId); + addCandidate(tier, modelId, value as AxisValue | undefined, { source: "provider", provider: name }); + } + } + return tier; +} + +// Bundled tiers never change at runtime; build them once per axis. +const staticTiers = new Map(); +function staticTiersFor(axis: Axis) { + let tiers = staticTiers.get(axis); + if (!tiers) { + tiers = { + registry: registryTier(axis), + vendor: metadataTier([...VENDORS].flatMap(listModelMetadata), "vendor", axis), + aggregator: metadataTier(AGGREGATORS.flatMap(listModelMetadata), "aggregator", axis), + }; + staticTiers.set(axis, tiers); + } + return tiers; +} + +/** + * Builds a same-name capability lookup for one configuration. Authority order per axis: built-in + * original-vendor registry, bundled vendor metadata, other configured providers' explicit + * declarations, bundled aggregator metadata. These are estimates of the model, not measurements of + * the gateway serving it; callers label them `inferred` and never let them outrank a declaration + * or a provider report. + */ +export function createModelCapabilityInference(config?: OcxConfig): (modelId: string, excludeProvider?: string) => InferredModelCapabilities { + const configured = new Map>(); + const configuredFor = (exclude: string | undefined, axis: Axis): Tier => { + const key = exclude ?? ""; + let byAxis = configured.get(key); + if (!byAxis) configured.set(key, byAxis = new Map()); + let tier = byAxis.get(axis); + if (!tier) byAxis.set(axis, tier = configuredTier(config, exclude, axis)); + return tier; + }; + return (modelId, excludeProvider) => { + const key = capabilityInferenceKey(modelId); + const result: InferredModelCapabilities = {}; + if (!key) return result; + for (const axis of ["contextWindow", "inputModalities", "reasoningEfforts"] as const) { + const tiers = staticTiersFor(axis); + for (const tier of [tiers.registry, tiers.vendor, configuredFor(excludeProvider, axis), tiers.aggregator]) { + if (!tier.has(key)) continue; + const found = tier.get(key); + if (found) (result as Record)[axis] = { value: Array.isArray(found.value) ? [...found.value] : found.value, basis: { ...found.basis } }; + break; + } + } + return result; + }; +} + +export function resetModelCapabilityInferenceForTests(): void { + staticTiers.clear(); +} diff --git a/src/providers/model-rename-fields.ts b/src/providers/model-rename-fields.ts index c2f47f33bfe..b805d2ff3ca 100644 --- a/src/providers/model-rename-fields.ts +++ b/src/providers/model-rename-fields.ts @@ -72,12 +72,14 @@ export const PROVIDER_MODEL_RENAME_ROLES = { modelInputModalities: "record", inputModalities: "none", modelCapabilities: "record", + modelCapabilityProbes: "record", modelMaxInputTokens: "record", modelAutoCompactTokenLimits: "record", defaultMaxOutputTokens: "none", minMaxOutputTokens: "none", modelMaxOutputTokens: "record", modelCosts: "record", + priceMultiplier: "none", autoReviewModel: "none", autoReviewModelOverrides: "record", headers: "none", diff --git a/src/server/auth-cors.ts b/src/server/auth-cors.ts index 7ea7487797e..0782b4bbb4a 100644 --- a/src/server/auth-cors.ts +++ b/src/server/auth-cors.ts @@ -14,6 +14,7 @@ import { retryOnResetPolicyConfigError, sanitizeModelCostsForDisplay, } from "../config"; +import { priceMultiplierConfigError } from "../config/schema/leaf-validators"; import { apiKeyTransportConfigError, projectContextConfigError, @@ -922,6 +923,7 @@ export function providerManagementConfigError( // modelCosts is a user-owned display overlay, not part of the canonical // forward seed; it is validated separately below (providerModelCostsConfigError). delete canonicalCandidate.modelCosts; + delete canonicalCandidate.priceMultiplier; // requestPacing is a user-owned transport overlay, not part of the canonical seed. delete canonicalCandidate.requestPacing; // retryOnReset is the same kind of overlay: it tunes how this provider's own Responses @@ -1023,6 +1025,8 @@ export function providerManagementConfigError( // it before it reaches the management API response (same rule as retryOn429 above). return `provider ${JSON.stringify(redactSecretString(name))} ${modelCostsError}`; } + const priceMultiplierError = priceMultiplierConfigError(raw.priceMultiplier); + if (priceMultiplierError) return `provider ${JSON.stringify(redactSecretString(name))} ${priceMultiplierError}`; const apiKeyTransportError = apiKeyTransportConfigError(typed); if (apiKeyTransportError) return `provider ${name} ${apiKeyTransportError}`; const projectContextError = projectContextConfigError(typed); @@ -1246,12 +1250,14 @@ const PROVIDER_CONFIG_FIELD_POLICY = { modelInputModalities: "editor", inputModalities: "editor", modelCapabilities: "editor", + modelCapabilityProbes: "editor", modelMaxInputTokens: "runtime", modelAutoCompactTokenLimits: "editor", defaultMaxOutputTokens: "editor", minMaxOutputTokens: "editor", modelMaxOutputTokens: "editor", modelCosts: "editor", + priceMultiplier: "editor", headers: "redacted", forwardClientHeaders: "editor", openRouterRouting: "editor", diff --git a/src/server/management/logs-usage-routes.ts b/src/server/management/logs-usage-routes.ts index a3aefdac65e..6f2c7e474dc 100644 --- a/src/server/management/logs-usage-routes.ts +++ b/src/server/management/logs-usage-routes.ts @@ -72,6 +72,7 @@ import { getRequestLogEntries, queryRequestLogs, type RequestLogEntry } from ".. import { decodeRequestLogCursor, selectRequestLogPoll } from "../request-log-cursor"; import { estimateComboCost, estimateRequestCost, normalizeCostTokens, tokensPerSecond } from "../../usage/cost"; import { userCostOverlayVersion } from "../../usage/user-cost-overlays"; +import { triggerPriceCatalogRefresh } from "../../usage/price-catalog"; import type { PersistedUsageAttempt } from "../../usage/log"; import { isAllowedRequestOrigin, jsonResponse, providerManagementConfigError, publicProviderBaseUrl, safeConfigDTO } from "../auth-cors"; import { applySystemEnvToggle } from "../system-env"; @@ -245,6 +246,7 @@ export async function handleLogsUsageRoutes(ctx: ManagementContext): Promise `probed` (a declaration written by the capability probe) > + * `reported` (discovery, registry or provider metadata) > `inferred` (a same-name estimate the + * catalog applies under `inferContextWindow`) > `unknown`. + */ +export type CapabilitySource = "manual" | "probed" | "reported" | "inferred" | "unknown"; + +export interface ModelCapabilitySources { + contextWindow: CapabilitySource; + inputModalities: CapabilitySource; + reasoningEfforts: CapabilitySource; +} + +export interface ModelCapabilitySourceFields { + capabilitySources: ModelCapabilitySources; + /** Same-name estimates for this model; present only when one exists. Display data, never routed. */ + inferredCapabilities?: InferredModelCapabilities; +} + +interface RowFacts { + contextWindowDeclared?: number; + inputModalitiesDeclared?: string[]; + reasoningOverridden?: boolean; + reasoningEfforts?: string[]; +} + +/** One lookup per `/api/models` listing; bundled tiers are shared across calls. */ +export function modelCapabilitySourceResolver(config: OcxConfig) { + const infer = createModelCapabilityInference(config); + return (row: Pick, facts: RowFacts): ModelCapabilitySourceFields => { + const probes = config.providers[row.provider]?.modelCapabilityProbes?.[row.id]; + const declared = (manual: boolean, probed: boolean | undefined): CapabilitySource | undefined => + manual ? (probed === true ? "probed" : "manual") : undefined; + const contextWindow = declared(facts.contextWindowDeclared !== undefined, false) + ?? (row.contextWindowInference ? "inferred" : typeof row.contextWindow === "number" && row.contextWindow > 0 ? "reported" : "unknown"); + const inputModalities = declared(facts.inputModalitiesDeclared !== undefined, probes?.inputModalities) + ?? (Array.isArray(row.inputModalities) && row.inputModalities.length > 0 ? "reported" : "unknown"); + const reasoningEfforts = declared(facts.reasoningOverridden === true, probes?.reasoningEfforts) + ?? (facts.reasoningEfforts !== undefined ? "reported" : "unknown"); + const inferred = infer(row.id, row.provider); + return { + capabilitySources: { contextWindow, inputModalities, reasoningEfforts }, + ...(Object.keys(inferred).length > 0 ? { inferredCapabilities: inferred } : {}), + }; + }; +} diff --git a/src/server/management/model-routes.ts b/src/server/management/model-routes.ts index a00a53b5a7c..1710e937d53 100644 --- a/src/server/management/model-routes.ts +++ b/src/server/management/model-routes.ts @@ -121,7 +121,10 @@ import { effectiveModelAliases, MODEL_ALIAS_PATTERN, templateAliasPlan } from ". import { aliasTemplateError } from "../../providers/alias-template"; import { isValidModelDiscoveryModelId } from "../../providers/model-discovery-limits"; import { commitProviderPatch } from "./provider-patch-transaction"; +import { pruneModelCapabilityProbes } from "../../config/provider-validation"; import { ConfigWritePublishedError } from "../../config/persist-unlocked"; +import { priceMultiplierConfigError } from "../../config/schema/leaf-validators"; +import { refreshPriceCatalogs, triggerPriceCatalogRefresh } from "../../usage/price-catalog"; import type { CatalogDisposition } from "../../codex/convergence-types"; import { comboPublicModelId } from "../../combos/types"; import { COMBO_NAMESPACE, comboDisabledModelSelectors, comboModelId, preservesPhysicalComboProvider } from "../../combos"; @@ -444,6 +447,7 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise key !== "multiplier")) { + return jsonResponse({ error: "only multiplier (number or null) is allowed" }, 400, req, config); + } + const multiplier = body.multiplier; + const validationError = multiplier === null ? null : priceMultiplierConfigError(multiplier ?? "invalid", "multiplier"); + if (validationError) return jsonResponse({ error: validationError }, 400, req, config); + const provider = config.providers[name]!; + const hadMultiplier = Object.hasOwn(provider, "priceMultiplier"); + const previous = provider.priceMultiplier; + // Sparse: null deletes the key, so an unset multiplier never appears in config.json. + if (multiplier === null) delete provider.priceMultiplier; + else provider.priceMultiplier = multiplier as number; + try { + persistConfig(config); + } catch (error) { + if (!(error instanceof ConfigWritePublishedError)) { + if (hadMultiplier) provider.priceMultiplier = previous; + else delete provider.priceMultiplier; + } + throw error; + } + return jsonResponse({ ok: true, provider: name, multiplier: multiplier as number | null }, 200, req, config); + } + + if (url.pathname === "/api/pricing/refresh" && req.method === "POST") { + // Forced full refresh of both public price lists; clears miss cooldowns. Per-source status + // carries a short error code and never a response body. + return jsonResponse({ ok: true, sources: await refreshPriceCatalogs({ force: true }) }, 200, req, config); + } + const displayNameMatch = url.pathname.match(/^\/api\/providers\/([^/]+)\/model-display-names$/); if (displayNameMatch && req.method === "PUT") { let name: string; @@ -1016,6 +1061,12 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise map !== undefined && Object.hasOwn(map, modelId)) diff --git a/src/server/management/model-row-price.ts b/src/server/management/model-row-price.ts new file mode 100644 index 00000000000..d0f8fac52e4 --- /dev/null +++ b/src/server/management/model-row-price.ts @@ -0,0 +1,30 @@ +import { resolveMatchedPrice } from "../../usage/cost"; + +/** Effective price shown on an `/api/models` row; absent on the row means the price is unknown. */ +export interface ModelRowPrice { + source: "manual" | "official" | "models-dev" | "openrouter"; + input: number; + output: number; + cacheRead: number; + cacheWrite: number; + /** ISO time of the catalog fetch that produced a models-dev / openrouter price. */ + fetchedAt?: string; + multiplier?: number; + apiEquivalent?: true; +} + +/** Effective (post-multiplier) rates and where they came from, via the same resolver the cost estimate uses. */ +export function modelRowPrice(provider: string, modelId: string): ModelRowPrice | undefined { + const price = resolveMatchedPrice(provider, modelId); + if (!price) return undefined; + const online = price.source === "models-dev" || price.source === "openrouter"; + const source: ModelRowPrice["source"] = price.source === "user" ? "manual" + : price.source === "models-dev" || price.source === "openrouter" ? price.source : "official"; + return { + source, + ...price.cost4, + ...(online && price.verifiedAt ? { fetchedAt: price.verifiedAt } : {}), + ...(price.multiplier !== undefined ? { multiplier: price.multiplier } : {}), + ...(price.apiEquivalent ? { apiEquivalent: true as const } : {}), + }; +} diff --git a/src/server/management/model-rows.ts b/src/server/management/model-rows.ts index ead4da078b9..339391237fa 100644 --- a/src/server/management/model-rows.ts +++ b/src/server/management/model-rows.ts @@ -8,6 +8,7 @@ * * Bodies are unchanged from their previous home; only `export` was added. */ +import { modelCapabilitySourceResolver, type ModelCapabilitySourceFields } from "./model-capability-sources"; import type { CatalogModel } from "../../codex/catalog"; import { antigravityFamilyDisabled } from "../../providers/antigravity-effort-families"; import { observeModelCacheRevision } from "../../codex/model-cache"; @@ -56,6 +57,7 @@ import { initialModelSelectionPending, pendingModelSelectionProviders } from ".. import { catalogFastRowEligible, fastRowId } from "../fast-row"; import { knownEffortRowIds } from "../effort-row"; import { visionSidecarConsumerClassifier } from "../../vision/eligibility"; +import { modelRowPrice, type ModelRowPrice } from "./model-row-price"; /** * One row of the `/api/models` list. Routed rows spread a `CatalogModel`, so the shape is @@ -72,6 +74,8 @@ export type ManagementModelRow = Partial & { custom?: boolean; customId?: string; manualPricing?: boolean; + /** Effective price and its source; absent when no tier can price the model. Never set on combos. */ + price?: ModelRowPrice; fastRowAvailable?: boolean; displayNameOverride?: string; displayNameSource?: "operator" | "provider" | "fallback"; @@ -99,6 +103,9 @@ export type ManagementModelRow = Partial & { * config — so this compares values against what the model would inherit anyway. */ reasoningOverridden?: boolean; + /** Per-axis provenance and same-name estimates (`model-capability-sources.ts`); routed rows only. */ + capabilitySources?: ModelCapabilitySourceFields["capabilitySources"]; + inferredCapabilities?: ModelCapabilitySourceFields["inferredCapabilities"]; /** * Tool support a canonical source positively asserts (native identity, a catalog row's * `"tools"` capability, or the parallel-call opt-in — the same evidence @@ -457,6 +464,7 @@ export async function listManagementModelRows( ), }; }); + const capabilitySources = modelCapabilitySourceResolver(config); const dedupedRouted = publicModels.map((m): ManagementModelRow | null => { // Codex-facing slug (one "/", slug-codec); disabledModels compares tolerate both forms. const namespaced = catalogModelSlug(m); @@ -476,6 +484,7 @@ export async function listManagementModelRows( : undefined; const inputModalitiesDeclared = routed ? declaredModelInputModalities(config, m.provider, m.id) : undefined; const contextWindowDeclared = routed ? config.providers[m.provider]?.modelContextWindows?.[m.id] : undefined; + const reasoningOverridden = routed ? reasoningOverrideFor(config, m.provider, m.id, catalogLadders) : undefined; return { ...m, ...displayName, @@ -489,7 +498,8 @@ export async function listManagementModelRows( // here, where the config and the registry are both in hand. ...(reasoningEfforts !== undefined ? { reasoningEfforts } : {}), ...(defaultReasoningEffort !== undefined ? { defaultReasoningEffort } : {}), - ...(routed ? { reasoningOverridden: reasoningOverrideFor(config, m.provider, m.id, catalogLadders) } : {}), + ...(routed ? { reasoningOverridden } : {}), + ...(routed ? capabilitySources(m, { contextWindowDeclared, inputModalitiesDeclared, reasoningOverridden, reasoningEfforts }) : {}), ...(inputModalitiesDeclared !== undefined ? { inputModalitiesDeclared } : {}), ...(contextWindowDeclared !== undefined ? { contextWindowDeclared } : {}), // The authoritative export default is the GATHERED row's own declared default — never @@ -534,6 +544,7 @@ export async function listManagementModelRows( const supportsReasoning = row.exportMetadata?.supportsReasoning ?? row.supportsReasoning ?? knownReasoningSupport(row); + const price = row.provider !== "combo" ? modelRowPrice(row.provider, row.id) : undefined; return { ...row, // One shared definition with the request path: the GUI must not re-derive consumer @@ -541,6 +552,7 @@ export async function listManagementModelRows( visionSidecarConsumer: isVisionSidecarConsumer(row.provider, row.id), ...(!row.native && modelCosts !== undefined && Object.hasOwn(modelCosts, row.id) ? { manualPricing: true } : {}), + ...(row.provider !== "combo" && price ? { price } : {}), ...(pending ? { disabled: true, initialSelectionPending: true } : {}), fastRowAvailable: !row.disabled && !pending && !knownIds.has(fastRowId(row.namespaced)) && catalogFastRowEligible(config, row), diff --git a/src/server/management/provider-capability-probe.ts b/src/server/management/provider-capability-probe.ts new file mode 100644 index 00000000000..a75f78d4e61 --- /dev/null +++ b/src/server/management/provider-capability-probe.ts @@ -0,0 +1,220 @@ +/** + * `POST /api/providers/probe-capabilities`: cheaply test what one model on one configured provider + * accepts, by reusing the single-request completion diagnostic with a body variant (an image part, + * or a reasoning effort). Probes run sequentially and never retry. Results carry codes and HTTP + * statuses only, never upstream bodies. With `apply: true`, conclusive results are written back as + * per-model declarations and labeled in `modelCapabilityProbes`. + * + * "Accepted" means the upstream did not refuse the request, not that it honored the field: a + * gateway that silently ignores `reasoning_effort` reads as accepting every effort. + */ +import type { OcxConfig, OcxProviderConfig } from "../../types"; +import { PROVIDER_REGISTRY } from "../../providers/registry"; +import { assessUrlDestination } from "../../lib/destination-policy"; +import { canonicalizeReasoningEfforts, isDeclaredReasoningEffort } from "../../reasoning-effort"; +import { isValidModelDiscoveryModelId } from "../../providers/model-discovery-limits"; +import { + mergeModelCapabilities, modelCapabilitiesConfigError, modelCapabilityProbesConfigError, pruneModelCapabilityProbes, +} from "../../config/provider-validation"; +import { saveConfigPreservingClaudeCode } from "../../config"; +import { ConfigWritePublishedError } from "../../config/persist-unlocked"; +import { clearModelCache } from "../../codex/model-cache"; +import { commitProviderPatch } from "./provider-patch-transaction"; +import { effectiveModelReasoningEfforts } from "./model-rows"; +import { jsonResponse } from "../auth-cors"; +import type { ManagementContext } from "./context"; +import { managementBodyTooLargeResponse, readManagementJsonBody } from "./body"; +import { + providerCompletionTestApplicability, runProviderCompletionTest, + type ProviderCompletionTestDependencies, type ProviderCompletionTestResult, +} from "./provider-completion-test"; + +export const CAPABILITY_PROBE_MAX_EFFORTS = 6; +const DEFAULT_EFFORTS = ["low", "medium", "high"]; +const BODY_KEYS = new Set(["name", "model", "checks", "efforts", "confirmMetered", "apply"]); +const LOCAL_DESTINATIONS = new Set(["localhost", "loopback", "private"]); +/** After one of these, every later probe would fail the same way; the rest are marked inconclusive. */ +const STOP_STATUSES = new Set([401, 403, 429]); +const STOP_CODES = new Set(["credential_missing", "credential_changed", "destination_refused", "aborted"]); + +type Check = "image" | "reasoning"; +type ProbeError = { code: string; status?: number }; +type Outcome = { status: "accepted" | "rejected" | "inconclusive"; error?: ProbeError; stop: boolean }; +export type ProviderCapabilityProbeResult = { + applicable: true; sent: true; metered: boolean; provider: string; model: string; + /** Plain prompt sent first; unless accepted, every variant is inconclusive (a 400 is not about the variant). */ + baseline: { status: Outcome["status"]; error?: ProbeError }; + image?: { status: Outcome["status"]; error?: ProbeError }; + reasoning?: { accepted: string[]; rejected: string[]; inconclusive: string[] }; + applied?: { inputModalities?: string[]; reasoningEfforts?: string[] }; +}; + +const invalid = (error = "invalid capability probe request") => jsonResponse({ error }, 400); + +/** API-key auth to a non-local destination is billed per request; everything else is not. */ +export function providerProbeIsMetered(name: string, row: OcxProviderConfig): boolean { + const authKind = PROVIDER_REGISTRY.find(entry => entry.id === name)?.authKind; + if ((row.authMode ?? authKind ?? "key") !== "key") return false; + const kind = typeof row.baseUrl === "string" ? assessUrlDestination(row.baseUrl)?.kind : undefined; + return kind === undefined || !LOCAL_DESTINATIONS.has(kind); +} + +function defaultEfforts(config: OcxConfig, name: string, model: string): string[] { + let known: string[] = []; + try { known = canonicalizeReasoningEfforts((effectiveModelReasoningEfforts(config, name, model) ?? []).filter(isDeclaredReasoningEffort)); } + catch { known = []; } + if (known.length > CAPABILITY_PROBE_MAX_EFFORTS) known = known.filter(effort => effort !== "none" && effort !== "minimal"); + return known.length > 0 ? known.slice(0, CAPABILITY_PROBE_MAX_EFFORTS) : [...DEFAULT_EFFORTS]; +} + +/** + * Completed means accepted; 400/422 means the upstream refused the variant. For the reasoning + * check `no_content` is also accepted: with a 32-token cap a reasoning model can spend the whole + * budget thinking and finish without a text delta, after the request itself was admitted. + */ +function classify(result: ProviderCompletionTestResult, check: Check): Outcome { + if (!result.applicable) return { status: "inconclusive", error: { code: result.reason }, stop: true }; + if (result.ok || (check === "reasoning" && result.error?.code === "no_content")) return { status: "accepted", stop: false }; + const code = result.error?.code ?? "network"; + const status = result.error?.status; + const error: ProbeError = { code, ...(status !== undefined ? { status } : {}) }; + if (code === "http_status" && (status === 400 || status === 422)) return { status: "rejected", error, stop: false }; + return { status: "inconclusive", error, stop: STOP_CODES.has(code) || (status !== undefined && STOP_STATUSES.has(status)) }; +} + +function parseBody(body: unknown, config: OcxConfig) { + if (!body || typeof body !== "object" || Array.isArray(body) || Object.keys(body).some(key => !BODY_KEYS.has(key))) return null; + const raw = body as Record; + if (typeof raw.name !== "string" || typeof raw.model !== "string") return null; + const model = raw.model.trim(); + if (!model || model.length > 256 || /[\x00-\x1f\x7f-\x9f]/.test(raw.model)) return null; + for (const flag of ["confirmMetered", "apply"] as const) if (raw[flag] !== undefined && typeof raw[flag] !== "boolean") return null; + let checks: Check[] = ["image", "reasoning"]; + if (raw.checks !== undefined) { + if (!Array.isArray(raw.checks) || raw.checks.length === 0 || raw.checks.some(check => check !== "image" && check !== "reasoning")) return null; + checks = (["image", "reasoning"] as const).filter(check => (raw.checks as unknown[]).includes(check)); + } + let efforts: string[] | undefined; + if (raw.efforts !== undefined) { + if (!checks.includes("reasoning") || !Array.isArray(raw.efforts) || raw.efforts.length === 0 + || raw.efforts.some(effort => typeof effort !== "string" || !isDeclaredReasoningEffort(effort))) return null; + efforts = canonicalizeReasoningEfforts(raw.efforts as string[]); + if (efforts.length > CAPABILITY_PROBE_MAX_EFFORTS) return null; + } + const apply = raw.apply === true; + // Applied results become per-model config keys, which have a stricter shape than a request model. + if (apply && (!isValidModelDiscoveryModelId(model) || ["__proto__", "prototype", "constructor"].includes(model))) return null; + const name = raw.name; + return { name, model, checks, efforts, apply, confirmMetered: raw.confirmMetered === true, known: Object.hasOwn(config.providers, name) && !!config.providers[name] }; +} + +function writeBack( + config: OcxConfig, name: string, model: string, applied: NonNullable, probedAt: string, +): void { + const live = config.providers[name]; + if (!live) throw new Error("provider removed during probe"); + const next: OcxProviderConfig = { ...live }; + const previous = live.modelCapabilityProbes && Object.hasOwn(live.modelCapabilityProbes, model) ? live.modelCapabilityProbes[model] : undefined; + const label = { probedAt, ...(previous?.inputModalities ? { inputModalities: true as const } : {}), + ...(previous?.reasoningEfforts ? { reasoningEfforts: true as const } : {}) }; + if (applied.inputModalities) { + next.modelCapabilities = mergeModelCapabilities(next.modelCapabilities, { [model]: { inputModalities: applied.inputModalities } }); + label.inputModalities = true; + } + if (applied.reasoningEfforts) { + next.modelReasoningEfforts = { ...(next.modelReasoningEfforts ?? {}), [model]: [...applied.reasoningEfforts] }; + // Same rule as PUT /api/model-settings: a stored default the new ladder cannot select is cleared. + const storedDefault = next.modelDefaultReasoningEfforts?.[model]; + if (typeof storedDefault === "string" && !applied.reasoningEfforts.includes(storedDefault)) { + const { [model]: _dropped, ...rest } = next.modelDefaultReasoningEfforts!; + if (Object.keys(rest).length > 0) next.modelDefaultReasoningEfforts = rest; + else delete next.modelDefaultReasoningEfforts; + } + label.reasoningEfforts = true; + } + next.modelCapabilityProbes = { ...(next.modelCapabilityProbes ?? {}), [model]: label }; + pruneModelCapabilityProbes(next); + if (modelCapabilitiesConfigError(next.modelCapabilities) || modelCapabilityProbesConfigError(next.modelCapabilityProbes)) { + throw new Error("probe results exceed the per-model declaration limits"); + } + config.providers[name] = next; +} + +const inFlight = new Set(); +export async function handleProviderCapabilityProbeRoute( + ctx: Pick, + probeDeps: ProviderCompletionTestDependencies = {}, +): Promise { + const { req, config, deps } = ctx; + let body: unknown; + try { body = await readManagementJsonBody(req); } catch (error) { + return managementBodyTooLargeResponse(error, req, config) ?? invalid(); + } + const parsed = parseBody(body, config); + if (!parsed) return invalid(); + if (!parsed.known) return jsonResponse({ error: "unknown provider" }, 404); + const { name, model, checks, apply } = parsed; + const blocked = providerCompletionTestApplicability(config, name, model); + if (blocked) return jsonResponse({ ...blocked, sent: false, provider: name, model }); + const efforts = checks.includes("reasoning") ? parsed.efforts ?? defaultEfforts(config, name, model) : []; + const plannedRequests = 1 + (checks.includes("image") ? 1 : 0) + efforts.length; + const metered = providerProbeIsMetered(name, config.providers[name]!); + if (metered && !parsed.confirmMetered) { + return jsonResponse({ applicable: true, sent: false, requiresConfirmation: true, metered: true, plannedRequests }); + } + if (inFlight.has(name)) return jsonResponse({ error: "capability probe already running for this provider" }, 409); + inFlight.add(name); + try { + let stopped = false; + const probe = async (check: Check, variant?: { image?: true; reasoningEffort?: string }) => { + const outcome = classify(await runProviderCompletionTest({ config, name, model, signal: req.signal, ...(variant ? { variant } : {}) }, probeDeps), check); + stopped = outcome.stop; + return outcome; + }; + const base = await probe("image"); + const result: ProviderCapabilityProbeResult = { applicable: true, sent: true, metered, provider: name, model, + baseline: { status: base.status, ...(base.error ? { error: base.error } : {}) } }; + if (base.status !== "accepted") stopped = true; + if (checks.includes("image")) { + const { status, error } = stopped ? { status: "inconclusive" as const, error: undefined } : await probe("image", { image: true }); + result.image = { status, ...(error ? { error } : {}) }; + } + if (checks.includes("reasoning")) { + const reasoning = { accepted: [] as string[], rejected: [] as string[], inconclusive: [] as string[] }; + for (const effort of efforts) { + if (stopped) { reasoning.inconclusive.push(effort); continue; } + reasoning[(await probe("reasoning", { reasoningEffort: effort })).status].push(effort); + } + result.reasoning = reasoning; + } + if (!apply) return jsonResponse(result); + + const applied: NonNullable = {}; + if (result.image?.status === "accepted") applied.inputModalities = ["text", "image"]; + else if (result.image?.status === "rejected") applied.inputModalities = ["text"]; + const reasoning = result.reasoning; + if (reasoning && reasoning.inconclusive.length === 0 && reasoning.accepted.length + reasoning.rejected.length > 0) { + applied.reasoningEfforts = canonicalizeReasoningEfforts(reasoning.accepted); + } + result.applied = applied; + if (!applied.inputModalities && !applied.reasoningEfforts) return jsonResponse(result); + let published = true; + try { + const probedAt = new Date().toISOString(); + commitProviderPatch(config, () => writeBack(config, name, model, applied, probedAt), + deps.saveConfigPreservingClaudeCode ?? saveConfigPreservingClaudeCode); + } catch (error) { + if (!(error instanceof ConfigWritePublishedError)) { + delete result.applied; + return jsonResponse({ ...result, error: "capability probe results could not be saved" }, 500); + } + published = false; + } + // Same order as PUT /api/model-settings: drop the cached rows the gather baked, then converge. + clearModelCache(name); + if (published) { try { await ctx.convergeCodexCatalog(); } catch { /* the saved config stays authoritative */ } } + return jsonResponse(result); + } finally { + inFlight.delete(name); + } +} diff --git a/src/server/management/provider-completion-test.ts b/src/server/management/provider-completion-test.ts index fbb559e08c0..9234516d402 100644 --- a/src/server/management/provider-completion-test.ts +++ b/src/server/management/provider-completion-test.ts @@ -58,7 +58,16 @@ export type ProviderCompletionTestResult = { termination: "completed" | "failed" | "timeout" | "aborted" | "too_large"; error: null | { code: Code; status?: number; message: string }; }; -export interface ProviderCompletionTestInput { config: OcxConfig; name: string; model: string; signal?: AbortSignal } +/** + * Optional request variant for the capability probe. It changes only the request body: an image + * part beside the prompt, or a reasoning-effort field. Sender, credentials and limits are shared. + */ +export interface ProviderCompletionTestVariant { image?: true; reasoningEffort?: string } +export interface ProviderCompletionTestInput { + config: OcxConfig; name: string; model: string; signal?: AbortSignal; variant?: ProviderCompletionTestVariant; +} +/** 1x1 transparent PNG; the smallest image every vision wire decodes. */ +export const COMPLETION_TEST_IMAGE_DATA_URL = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="; export interface ProviderCompletionTestDependencies { now?: () => number; timeoutMs?: number; @@ -115,20 +124,36 @@ function abortable(promise: Promise, signal: AbortSignal): Promise { }); } class DiagnosticFailure extends Error { constructor(readonly code: Code) { super(MESSAGES[code]); } } - -/** Direct diagnostic; callers supply an existing provider name and an already validated model. */ -export async function runProviderCompletionTest( - { config, name, model, signal: requestSignal }: ProviderCompletionTestInput, - deps: ProviderCompletionTestDependencies = {}, -): Promise { +function requestBody(wire: Wire, model: string, providerRouting: unknown, variant: ProviderCompletionTestVariant = {}): string { + const effort = variant.reasoningEffort; + if (wire === "openai-chat") { + const content = variant.image ? [{ type: "text", text: COMPLETION_TEST_PROMPT }, + { type: "image_url", image_url: { url: COMPLETION_TEST_IMAGE_DATA_URL } }] : COMPLETION_TEST_PROMPT; + return JSON.stringify({ model, messages: [{ role: "user", content }], stream: true, max_tokens: 32, + ...(effort !== undefined ? { reasoning_effort: effort } : {}), ...(providerRouting ? { provider: providerRouting } : {}) }); + } + const input = variant.image ? [{ role: "user", content: [{ type: "input_text", text: COMPLETION_TEST_PROMPT }, + { type: "input_image", image_url: COMPLETION_TEST_IMAGE_DATA_URL }] }] : COMPLETION_TEST_PROMPT; + return JSON.stringify({ model, input, stream: true, max_output_tokens: 32, store: false, + ...(effort !== undefined ? { reasoning: { effort } } : {}) }); +} +type Inapplicable = Extract; +/** The no-send applicability gate, shared with the capability probe. Never dereferences credentials. */ +export function providerCompletionTestApplicability(config: OcxConfig, name: string, model: string): Inapplicable | null { + const gate = applicability(config, name, model); + return gate.blocked ?? null; +} +type Gate = { blocked: Inapplicable } | { blocked?: undefined; row: OcxProviderConfig; snapshot: OcxProviderConfig; + native: string; wire: Wire; destinationRefused: boolean }; +function applicability(config: OcxConfig, name: string, model: string): Gate { const row = config.providers[name]; if (!Object.hasOwn(config.providers, name) || !row) throw new Error("unknown provider"); - if (row.disabled) return { applicable: false, reason: "disabled" }; + if (row.disabled) return { blocked: { applicable: false, reason: "disabled" } }; const authKind = PROVIDER_REGISTRY.find(entry => entry.id === name)?.authKind; - if (row.authMode === "forward" || authKind === "forward") return { applicable: false, reason: "forward_auth" }; - if (row.authMode === "oauth" || authKind === "oauth") return { applicable: false, reason: "oauth_auth" }; + if (row.authMode === "forward" || authKind === "forward") return { blocked: { applicable: false, reason: "forward_auth" } }; + if (row.authMode === "oauth" || authKind === "oauth") return { blocked: { applicable: false, reason: "oauth_auth" } }; const detail = transportExclusion(row); - if (detail) return { applicable: false, reason: "unsupported_transport", detail }; + if (detail) return { blocked: { applicable: false, reason: "unsupported_transport", detail } }; // Metadata resolution must never dereference keychain/env/pool credentials. const { apiKey: _key, apiKeyPool: _pool, _apiKeyAttempt: _attempt, ...metadata } = row; let snapshot: OcxProviderConfig = metadata; @@ -136,7 +161,20 @@ export async function runProviderCompletionTest( try { snapshot = routedProviderConfig(name, metadata); } catch { destinationRefused = true; } const native = nativeModel(config, name, snapshot, model); const wire = resolveWireProtocolOverride(name, native, snapshot).adapter; - if (wire !== "openai-chat" && wire !== "openai-responses") return { applicable: false, reason: "unsupported_adapter", adapter: wire }; + if (wire !== "openai-chat" && wire !== "openai-responses") { + return { blocked: { applicable: false, reason: "unsupported_adapter", adapter: wire } }; + } + return { row, snapshot, native, wire: wire as Wire, destinationRefused }; +} + +/** Direct diagnostic; callers supply an existing provider name and an already validated model. */ +export async function runProviderCompletionTest( + { config, name, model, signal: requestSignal, variant }: ProviderCompletionTestInput, + deps: ProviderCompletionTestDependencies = {}, +): Promise { + const gate = applicability(config, name, model); + if (gate.blocked) return gate.blocked; + const { row, snapshot, native, wire, destinationRefused } = gate; const now = deps.now ?? (() => performance.now()); const result: Extract = { applicable: true, ok: false, provider: name, requestedModel: model, @@ -176,10 +214,7 @@ export async function runProviderCompletionTest( }; try { signal.throwIfAborted(); - const body = JSON.stringify(authority.wire === "openai-chat" - ? { model: authority.wireModel, messages: [{ role: "user", content: COMPLETION_TEST_PROMPT }], stream: true, max_tokens: 32, - ...(authority.providerRouting ? { provider: authority.providerRouting } : {}) } - : { model: authority.wireModel, input: COMPLETION_TEST_PROMPT, stream: true, max_output_tokens: 32, store: false }); + const body = requestBody(authority.wire, authority.wireModel, authority.providerRouting, variant); const pending = providerOutboundPost(name, { ...authority.routed, allowPrivateNetwork: authority.allowLocalCleartextPost }, authority.url, { headers: authority.headers, body, signal }, { ...deps.outboundDependencies, allowLocalCleartextPost: authority.allowLocalCleartextPost, diff --git a/src/server/management/provider-overwrite-carry.ts b/src/server/management/provider-overwrite-carry.ts index e0afd05f5cc..8321d69d5c0 100644 --- a/src/server/management/provider-overwrite-carry.ts +++ b/src/server/management/provider-overwrite-carry.ts @@ -1,4 +1,4 @@ -import { nonBlankStringArrayConfigError, normalizeNonBlankStringArray } from "../../config/provider-validation"; +import { modelCapabilityProbesConfigError, nonBlankStringArrayConfigError, normalizeNonBlankStringArray } from "../../config/provider-validation"; import { retryOn429PolicyConfigError, retryOnResetPolicyConfigError, @@ -197,6 +197,8 @@ export function providerCompatFieldConfigError(provider: Record const promoteAdditionalTools = provider.promoteAdditionalTools; if (promoteAdditionalTools !== undefined && typeof promoteAdditionalTools !== "boolean") return "promoteAdditionalTools must be a boolean"; if (provider.forwardAdvisorTool !== undefined && typeof provider.forwardAdvisorTool !== "boolean") return "forwardAdvisorTool must be a boolean"; + const probesError = modelCapabilityProbesConfigError(provider.modelCapabilityProbes); + if (probesError) return probesError; const wire = provider.reasoningWireFormat; if (wire !== undefined && !PROVIDER_REASONING_WIRE_FORMATS.includes(wire as never)) { return `reasoningWireFormat must be one of: ${PROVIDER_REASONING_WIRE_FORMATS.join(", ")}`; diff --git a/src/server/management/provider-routes.ts b/src/server/management/provider-routes.ts index 758c8e34c75..ae1ae5b5b61 100644 --- a/src/server/management/provider-routes.ts +++ b/src/server/management/provider-routes.ts @@ -1,4 +1,4 @@ -import { contextTierRecordConfigError, modelCapabilitiesConfigError, mergeModelCapabilities } from "../../config/provider-validation"; +import { carryModelCapabilityProbes, contextTierRecordConfigError, dropPatchedModelCapabilityProbes, modelCapabilitiesConfigError, mergeModelCapabilities } from "../../config/provider-validation"; import { DECLARABLE_HOSTED_TOOL_TYPES } from "../../responses/hosted-tool-policy"; import { randomUUID } from "node:crypto"; import { cleanupRemovedGcpCredentials, providerGcpCredentialMarkers, snapshotProviderGcpCredentialMarkers } from "../../providers/gcp-credential-cleanup"; @@ -530,6 +530,7 @@ function applyProviderPatchFields( const capabilities = mergeModelCapabilities(next.modelCapabilities, rawBody.modelCapabilities); if (capabilities === undefined) delete next.modelCapabilities; else next.modelCapabilities = capabilities; + dropPatchedModelCapabilityProbes(next, rawBody.modelCapabilities); touched = true; } if (Object.hasOwn(rawBody, "modelContextTiers")) { @@ -908,7 +909,7 @@ export async function handleProviderRoutes(ctx: ManagementContext): Promise>, test: (price: MatchedPrice) => boolean): boolean { + return (estimate.price !== undefined && test(estimate.price)) || estimate.attempts?.some(a => test(a.price)) === true; +} + /** Display-time cost estimate for one log entry (or its attempt list), including the reasons that qualify the estimate. */ export function costResult(entry: MetricSource): CostResult { const tier = serviceTierContext(entry); @@ -220,6 +227,10 @@ export function costResult(entry: MetricSource): CostResult { ? "expected_price_overlay" as const : undefined, estimate.price?.source === "user" || estimate.attempts?.some(a => a.price.source === "user") ? "provider_cost_overlay" as const : undefined, + anyPrice(estimate, price => price.source === "models-dev" || price.source === "openrouter") + ? "online_price_catalog" as const : undefined, + anyPrice(estimate, price => price.multiplier !== undefined) ? "relay_price_multiplier" as const : undefined, + anyPrice(estimate, price => price.apiEquivalent === true) ? "subscription_api_equivalent" as const : undefined, estimate.priorityLowerBound ? "priority_lower_bound" as const : undefined, ].filter((reason): reason is CostEstimateReason => reason !== undefined); return { kind: "value", estimate, estimateReasons }; diff --git a/src/types.ts b/src/types.ts index f826d6f2116..a738ff9b580 100644 --- a/src/types.ts +++ b/src/types.ts @@ -143,6 +143,7 @@ export type { TierDecision, OcxProviderConfig, ModelCapabilities, + ModelCapabilityProbeRecord, ReasoningCapabilities, } from "./types/provider"; diff --git a/src/types/provider.ts b/src/types/provider.ts index d2bd1e82845..4ddb1077075 100644 --- a/src/types/provider.ts +++ b/src/types/provider.ts @@ -305,6 +305,13 @@ export interface ReasoningCapabilities { defaultThinkingEnabled?: boolean; } +/** Which per-model declarations a capability probe wrote, and when. Only `true` flags are stored. */ +export interface ModelCapabilityProbeRecord { + probedAt: string; + inputModalities?: true; + reasoningEfforts?: true; +} + /** Explicit per-model operator declarations; absent axes keep legacy behavior. */ export interface ModelCapabilities extends ReasoningCapabilities { inputModalities?: Array<"text" | "image" | "audio" | "video">; @@ -726,6 +733,13 @@ export interface OcxProviderConfig { */ inputModalities?: string[]; modelCapabilities?: Record; + /** + * Provenance for per-model declarations written by `POST /api/providers/probe-capabilities`. + * An axis flag means that model's `modelCapabilities[model].inputModalities` or + * `modelReasoningEfforts[model]` came from a live probe at `probedAt` (ISO 8601). A manual edit of + * that axis drops its flag, and a flag never outlives the declaration it labels; absent when empty. + */ + modelCapabilityProbes?: Record; /** Model-specific max input token limits. Values cap auto_compact_token_limit. */ modelMaxInputTokens?: Record; /** @@ -751,6 +765,13 @@ export interface OcxProviderConfig { * all-zero entry means "not billable here" and falls through to the catalogs. */ modelCosts?: Record; + /** + * Opt-in relay multiplier (finite, > 0, <= 1000) applied to every automatic price + * (official, models.dev, OpenRouter) for this provider in the `~$` estimate, for a relay that + * charges a markup or discount over the vendor's list price. Never applied to `modelCosts`. + * Absent means 1; the stored form is sparse. + */ + priceMultiplier?: number; /** * Provider-wide auto-review (approval) model for routed models of this provider. * diff --git a/src/usage/cost.ts b/src/usage/cost.ts index 809aba5b87a..c3bb3288d90 100644 --- a/src/usage/cost.ts +++ b/src/usage/cost.ts @@ -6,7 +6,9 @@ * - matching is exact native provider/model ID via the jawcode alias table * (no fuzzy, no case-fold, no resolvedModel/requestedModel fallback); * - user-configured provider overlay wins, then jawcode nonzero price, then the expected-price overlay - * (verified / verified-derived only), otherwise null => UI shows em dash; + * (verified / verified-derived only), then the online catalogs (models.dev, OpenRouter; always + * estimates), otherwise null => UI shows em dash; + * - a provider's opt-in `priceMultiplier` scales every non-user price, never a user price; * - combo sums per-attempt costs and fails closed when any attempt is unpriced. */ import { @@ -20,7 +22,8 @@ import { canonicalFastTierMarker } from "../providers/fastwire"; import { baseProviderLabel } from "../providers/label"; import type { PersistedUsageAttempt, UsageStatus } from "./log"; import { canonicalAntigravityUsageModel } from "../providers/antigravity-models"; -import { activeAccountPricingProviders, activeConfiguredProviders, activeUserCostOverlays, userCostOverlayVersion } from "./user-cost-overlays"; +import { activeAccountPricingProviders, activeConfiguredProviders, activePriceMultipliers, activeSubscriptionProviders, activeUserCostOverlays, userCostOverlayVersion } from "./user-cost-overlays"; +import { resolveCatalogPrice } from "./price-catalog"; import { EXPECTED_PRICE_OVERLAYS, findExpectedPriceOverlay, @@ -83,10 +86,14 @@ export interface MatchedPrice { jawcodeProvider?: string; cost4: Cost4; promptLengthPricing?: PromptLengthPricing; - source: "jawcode" | "expected" | "user"; + source: "jawcode" | "expected" | "user" | "models-dev" | "openrouter"; sourceRef?: string; verifiedAt?: string; status: "verified" | "verified-derived"; + /** Provider `priceMultiplier` already folded into `cost4` (relay markup/discount). */ + multiplier?: number; + /** Subscription-billed provider: the price is the API-equivalent, not what the plan charges. */ + apiEquivalent?: true; } export interface AttemptCostEstimate { @@ -212,6 +219,7 @@ export function resolveMatchedPrice( userOverlays: readonly ExpectedPriceOverlay[] = activeUserCostOverlays(), options: PriceResolutionOptions = {}, ): MatchedPrice | null { + const configuredName = provider; // Literal configured providers win over account identities. Only then use // config-owned Codex identities, followed by the existing historical suffix // grammar. Never infer an account by stripping an arbitrary suffix. @@ -231,14 +239,40 @@ export function resolveMatchedPrice( // user overlays get a NEW array identity + version bump on every config refresh, // so memoized rows never go stale. if (overlays === EXPECTED_PRICE_OVERLAYS && userOverlays === activeUserCostOverlays()) { - const cacheKey = `${userCostOverlayVersion()} ${options.allowModelLevelFallback !== false} ${provider} ${modelId}`; + // userCostOverlayVersion() also folds in the online catalog version. + // The decorated price is memoized (stable identity for callers that compare it); the + // configured name is part of the key because the multiplier and subscription label follow it, + // not the collapsed namespace. The version covers both sets. + const cacheKey = `${userCostOverlayVersion()} ${options.allowModelLevelFallback !== false} ${configuredName} ${provider} ${modelId}`; if (!priceMemo.has(cacheKey)) { if (priceMemo.size >= 512) priceMemo.clear(); - priceMemo.set(cacheKey, resolveMatchedPriceInner(provider, modelId, overlays, userOverlays, options)); + priceMemo.set(cacheKey, decoratePrice(resolveMatchedPriceInner(provider, modelId, overlays, userOverlays, options), configuredName, provider)); } return priceMemo.get(cacheKey)!; } - return resolveMatchedPriceInner(provider, modelId, overlays, userOverlays, options); + return decoratePrice(resolveMatchedPriceInner(provider, modelId, overlays, userOverlays, options), configuredName, provider); +} + +function scaleCost4(cost4: Cost4, k: number): Cost4 { + return { input: cost4.input * k, output: cost4.output * k, cacheRead: cost4.cacheRead * k, cacheWrite: cost4.cacheWrite * k }; +} + +/** + * Relay multiplier and subscription label, applied to non-user prices only. The multiplier is + * keyed by the literal configured provider name (before account-namespace collapse); the + * subscription set also holds exact account identities. + */ +function decoratePrice(price: MatchedPrice | null, configuredName: string, namespace: string): MatchedPrice | null { + if (!price || price.source === "user") return price; + const multiplier = activePriceMultipliers().get(configuredName); + const subscription = activeSubscriptionProviders(); + const apiEquivalent = subscription.has(configuredName) || subscription.has(namespace); + if (multiplier === undefined && !apiEquivalent) return price; + return { + ...price, + ...(multiplier !== undefined ? { cost4: scaleCost4(price.cost4, multiplier), multiplier } : {}), + ...(apiEquivalent ? { apiEquivalent: true as const } : {}), + }; } const priceMemo = new Map(); @@ -317,9 +351,8 @@ function resolveMatchedPriceExact( if (!overlay || !validCost4(overlay.cost4) || !hasNonZeroCost(overlay.cost4)) { if (options.allowModelLevelFallback === false) return null; const fallback = resolveModelLevelPrice(provider, modelId); - return fallback - ? { ...fallback, cost4: multiplyCursorFastCost(fallback.cost4, provider, modelId), ...cursorFastPriceProvenance(provider, modelId) } - : null; + if (fallback) return { ...fallback, cost4: multiplyCursorFastCost(fallback.cost4, provider, modelId), ...cursorFastPriceProvenance(provider, modelId) }; + return resolveOnlineCatalogPrice(provider, modelId); } if (overlay.status === "unverified") return null; return { @@ -351,6 +384,21 @@ function cursorFastPriceProvenance(provider: string, modelId: string): Pick/price-catalog-cache.json` + * and read synchronously, so a lookup never touches the network. A source that cannot be fetched + * keeps its previous snapshot. A manually set price and a verified official price outrank every + * row here; a catalog price is always an estimate and carries its source. + * + * Refresh paths (all background, never on a request path): + * - daily: a source older than 24h is replaced wholesale (`refreshPriceCatalogs`); + * - miss-driven: a model no tier could price is queued by `notePriceCatalogMiss`. The feeds are + * whole-table GETs (models.dev has no per-model endpoint), so one miss costs one request per + * source, but only the rows for the queued keys are MERGED into the existing snapshot; + * full replacement stays with the daily refresh. A key still unfound afterwards cools down for + * 6h, and a source gets at most one miss-driven fetch per cooldown window. + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { atomicWriteFile } from "../config/atomic-write"; +import { getConfigDir } from "../config/paths"; +import { readBoundedResponseBytes } from "../lib/bounded-body"; +import { trackBackgroundWork } from "../lib/background-work"; +import { isPassiveRead } from "../lib/passive-read"; +import { capabilityInferenceKey } from "../providers/model-capability-inference"; +import type { Cost4 } from "./expected-prices"; +import { bumpPriceCatalogVersion } from "./price-catalog-version"; +import { validCost4 } from "./user-cost-validation"; + +export { priceCatalogVersion } from "./price-catalog-version"; + +export type CatalogPriceSource = "models-dev" | "openrouter"; +export interface CatalogPrice { + source: CatalogPriceSource; + cost4: Cost4; + vendor: string; + vendorModelId: string; + fetchedAt: number; + url: string; +} +export interface CatalogSourceStatus { ok: boolean; fetchedAt?: string; count?: number; error?: string } + +interface Row { cost4: Cost4; vendor: string; vendorModelId: string; fetchedAt?: number } +interface Snapshot { fetchedAt: number; prices: Map } +interface CatalogFile { + version: 1; + sources: Partial }>>; + missCooldown?: Record; +} +interface SourceDef { id: CatalogPriceSource; url: string; parse(raw: unknown): Map } + +export const PRICE_CATALOG_TTL_MS = 24 * 60 * 60 * 1000; +export const PRICE_CATALOG_MISS_COOLDOWN_MS = 6 * 60 * 60 * 1000; +const FAILURE_BACKOFF_MS = 60_000; +const FETCH_TIMEOUT_MS = 15_000; +const MAX_FEED_BYTES = 32 * 1024 * 1024; +const MAX_ROWS = 20_000; +const FILENAME = "price-catalog-cache.json"; + +/** Original vendors only: a reseller's listing is not the model's own price. */ +const MODELS_DEV_VENDORS = ["anthropic", "openai", "google", "deepseek", "moonshotai", "minimax", "zai", "xai", "mistral", "xiaomi", "stepfun", "alibaba"]; + +/** Same-model key: `cn:` channel prefix, `vendor/` prefix, case, dot-vs-hyphen and `[...]` suffix are ignored. */ +export function priceCatalogKey(modelId: string): string { + return capabilityInferenceKey(modelId.replace(/^[a-z0-9_-]+:/i, "")); +} + +const record = (v: unknown): Record | undefined => + v !== null && typeof v === "object" && !Array.isArray(v) ? v as Record : undefined; +const rate = (v: unknown): number | undefined => { + const n = typeof v === "string" && v.trim() ? Number(v) : v; + return typeof n === "number" && Number.isFinite(n) && n >= 0 ? n : undefined; +}; + +/** A key claimed by two rows with different prices answers nothing (the `addCandidate` rule of #8674). */ +function addRow(out: Map, key: string, row: Row): void { + if (!key) return; + const existing = out.get(key); + if (existing === undefined) out.set(key, row); + else if (existing !== null && JSON.stringify(existing.cost4) !== JSON.stringify(row.cost4)) out.set(key, null); +} +function settle(out: Map): Map { + const kept = new Map(); + for (const [key, row] of out) if (row && kept.size < MAX_ROWS) kept.set(key, row); + return kept; +} +function priced(input?: number, output?: number, cacheRead?: number, cacheWrite?: number): Cost4 | undefined { + if (input === undefined || output === undefined || (input === 0 && output === 0)) return undefined; + const cost4 = { input, output, cacheRead: cacheRead ?? 0, cacheWrite: cacheWrite ?? 0 }; + return validCost4(cost4) ? cost4 : undefined; +} + +const SOURCES: readonly SourceDef[] = [ + { + id: "models-dev", + url: "https://models.dev/api.json", + parse(raw) { + const out = new Map(); + const root = record(raw); + for (const vendor of MODELS_DEV_VENDORS) { + for (const [id, value] of Object.entries(record(record(root?.[vendor])?.models) ?? {})) { + const cost = record(record(value)?.cost); + const cost4 = priced(rate(cost?.input), rate(cost?.output), rate(cost?.cache_read), rate(cost?.cache_write)); + if (cost4 && id.length <= 256) addRow(out, priceCatalogKey(id), { cost4, vendor, vendorModelId: id }); + } + } + return settle(out); + }, + }, + { + id: "openrouter", + url: "https://openrouter.ai/api/v1/models", + parse(raw) { + const out = new Map(); + const data = record(raw)?.data; + const perMillion = (v: unknown) => { const n = rate(v); return n === undefined ? undefined : n * 1_000_000; }; + for (const value of Array.isArray(data) ? data : []) { + const row = record(value); + const id = row?.id; + if (typeof id !== "string" || id.length > 256 || id.includes(":") || id.startsWith("openrouter/") || !id.includes("/")) continue; + const p = record(row?.pricing); + const cost4 = priced(perMillion(p?.prompt), perMillion(p?.completion), perMillion(p?.input_cache_read), perMillion(p?.input_cache_write)); + if (cost4) addRow(out, priceCatalogKey(id), { cost4, vendor: id.slice(0, id.indexOf("/")), vendorModelId: id }); + } + return settle(out); + }, + }, +]; + +interface State { + snapshots: Map; + inFlight: Map }>; + retryAfter: Map; + lastMissFetch: Map; + cooldown: Map; + pendingMiss: Set; + missScheduled: boolean; +} +const states = new Map(); +let autoEnabled = false; +let testFetch: typeof fetch | null = null; + +function loadFile(dir: string, s: State): void { + try { + const file = record(JSON.parse(readFileSync(join(dir, FILENAME), "utf8"))); + if (file?.version !== 1) return; + for (const def of SOURCES) { + const src = record(record(file.sources)?.[def.id]); + if (!src || typeof src.fetchedAt !== "number" || !Number.isFinite(src.fetchedAt)) continue; + const prices = new Map(); + for (const [key, v] of Object.entries(record(src.prices) ?? {})) { + const r = record(v); + const c = r?.cost4 as Cost4 | undefined; + if (prices.size >= MAX_ROWS || !r || typeof r.vendor !== "string" || typeof r.vendorModelId !== "string" || !validCost4(c)) continue; + prices.set(key, { + cost4: { input: c.input, output: c.output, cacheRead: c.cacheRead, cacheWrite: c.cacheWrite }, + vendor: r.vendor, + vendorModelId: r.vendorModelId, + ...(typeof r.fetchedAt === "number" ? { fetchedAt: r.fetchedAt } : {}), + }); + } + s.snapshots.set(def.id, { fetchedAt: src.fetchedAt, prices }); + } + for (const [key, until] of Object.entries(record(file.missCooldown) ?? {})) { + if (typeof until === "number" && until > Date.now()) s.cooldown.set(key, until); + } + } catch { /* Missing or corrupt cache: no catalog, today's behavior. */ } +} + +function stateFor(dir: string): State { + let s = states.get(dir); + if (!s) { + s = { snapshots: new Map(), inFlight: new Map(), retryAfter: new Map(), lastMissFetch: new Map(), cooldown: new Map(), pendingMiss: new Set(), missScheduled: false }; + states.set(dir, s); + loadFile(dir, s); + } + return s; +} + +function persist(dir: string, s: State): void { + const file: CatalogFile = { version: 1, sources: {} }; + for (const [id, snap] of s.snapshots) file.sources[id] = { fetchedAt: snap.fetchedAt, prices: Object.fromEntries(snap.prices) }; + const now = Date.now(); + const live = [...s.cooldown].filter(([, until]) => until > now).slice(-500); + if (live.length) file.missCooldown = Object.fromEntries(live); + try { atomicWriteFile(join(dir, FILENAME), JSON.stringify(file) + "\n"); } catch { /* Memory still serves lookups. */ } +} + +/** Synchronous lookup: the first source in registry order that holds the key. */ +export function lookupCatalogPrice(modelId: string, dir = getConfigDir()): CatalogPrice | null { + const key = priceCatalogKey(modelId); + if (!key) return null; + const s = stateFor(dir); + for (const def of SOURCES) { + const snap = s.snapshots.get(def.id); + const row = snap?.prices.get(key); + if (snap && row) { + return { source: def.id, cost4: row.cost4, vendor: row.vendor, vendorModelId: row.vendorModelId, fetchedAt: row.fetchedAt ?? snap.fetchedAt, url: def.url }; + } + } + return null; +} + +/** Lookup for the resolver: on a miss, queue a background fetch for this key. Never blocks. */ +export function resolveCatalogPrice(modelId: string): CatalogPrice | null { + const found = lookupCatalogPrice(modelId); + if (!found) notePriceCatalogMiss(modelId); + return found; +} + +function autoAllowed(): boolean { + return (testFetch !== null || process.env["NODE_ENV"] !== "test") && !isPassiveRead(); +} + +/** Record a key no tier could price and schedule one coalesced background refresh. */ +export function notePriceCatalogMiss(modelId: string): void { + if (!autoEnabled || !autoAllowed()) return; + const key = priceCatalogKey(modelId); + const dir = getConfigDir(); + const s = stateFor(dir); + if (!key || s.pendingMiss.has(key) || (s.cooldown.get(key) ?? 0) > Date.now()) return; + s.pendingMiss.add(key); + if (s.missScheduled) return; + s.missScheduled = true; + setTimeout(() => { trackBackgroundWork("price-catalog-miss", flushMisses(dir, s).catch(() => undefined)); }, 0); +} + +async function flushMisses(dir: string, s: State): Promise { + const keys = new Set(s.pendingMiss); + s.pendingMiss.clear(); + s.missScheduled = false; + const now = Date.now(); + let fetched = false; + for (const def of SOURCES) { + const snap = s.snapshots.get(def.id); + const due = !snap || now - snap.fetchedAt >= PRICE_CATALOG_TTL_MS; + if (!due && now - (s.lastMissFetch.get(def.id) ?? 0) < PRICE_CATALOG_MISS_COOLDOWN_MS) continue; + const status = await refreshSource(dir, s, def, due ? { full: true } : { full: false, keys }); + if (status.ok) { + fetched = true; + if (!due) s.lastMissFetch.set(def.id, Date.now()); + } + } + if (!fetched) return; + const until = Date.now() + PRICE_CATALOG_MISS_COOLDOWN_MS; + for (const key of keys) if (!lookupCatalogPrice(key, dir)) s.cooldown.set(key, until); + persist(dir, s); +} + +async function fetchFeed(def: SourceDef): Promise> { + const signal = AbortSignal.timeout(FETCH_TIMEOUT_MS); + const response = await (testFetch ?? fetch)(def.url, { + headers: { accept: "application/json", "user-agent": "opencodex-price-catalog" }, + redirect: "error", + signal, + }); + if (!response.ok) throw new Error(`http_${response.status}`); + const body = await readBoundedResponseBytes(response, { maxBytes: MAX_FEED_BYTES, signal }); + if (body.oversized) throw new Error("too_large"); + const parsed = def.parse(JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(body.bytes))); + if (parsed.size === 0) throw new Error("empty"); + return parsed; +} + +function statusOf(s: State, def: SourceDef): CatalogSourceStatus { + const snap = s.snapshots.get(def.id); + return snap ? { ok: true, fetchedAt: new Date(snap.fetchedAt).toISOString(), count: snap.prices.size } : { ok: true, count: 0 }; +} + +type Mode = { full: true; force?: boolean } | { full: false; keys: ReadonlySet }; + +async function refreshSource(dir: string, s: State, def: SourceDef, mode: Mode): Promise { + const flight = s.inFlight.get(def.id); + if (flight) { + const joined = await flight.promise; + if (flight.full || !mode.full) return joined; + } + const now = Date.now(); + const snap = s.snapshots.get(def.id); + const forced = mode.full && mode.force === true; + if (!forced && now < (s.retryAfter.get(def.id) ?? 0)) return { ok: false, error: "backoff" }; + if (mode.full && !forced && snap && now - snap.fetchedAt < PRICE_CATALOG_TTL_MS) return statusOf(s, def); + const promise = (async (): Promise => { + try { + const parsed = await fetchFeed(def); + const at = Date.now(); + const before = s.snapshots.get(def.id); + if (mode.full) { + s.snapshots.set(def.id, { fetchedAt: at, prices: parsed }); + bumpPriceCatalogVersion(); + } else { + const next = new Map(before?.prices ?? []); + let added = 0; + for (const key of mode.keys) { + const row = parsed.get(key); + if (row && !next.has(key)) { + next.set(key, { ...row, fetchedAt: at }); + added++; + } + } + // No snapshot yet: fetchedAt 0 keeps the source due for its full daily refresh. + if (added) { + s.snapshots.set(def.id, { fetchedAt: before?.fetchedAt ?? 0, prices: next }); + bumpPriceCatalogVersion(); + } + } + persist(dir, s); + return statusOf(s, def); + } catch (error) { + s.retryAfter.set(def.id, Date.now() + FAILURE_BACKOFF_MS); + const message = error instanceof Error ? error.message : ""; + const code = /^(http_\d+|too_large|empty)$/.test(message) ? message : error instanceof SyntaxError ? "bad_json" : "fetch_failed"; + return { ok: false, error: code }; + } + })(); + s.inFlight.set(def.id, { full: mode.full, promise }); + try { return await promise; } finally { if (s.inFlight.get(def.id)?.promise === promise) s.inFlight.delete(def.id); } +} + +/** + * Daily refresh of every source: skip a fresh one unless `force`, coalesce concurrent callers, + * keep the previous snapshot when a fetch fails. `force` also clears miss cooldowns. + */ +export async function refreshPriceCatalogs(options: { force?: boolean; configDir?: string } = {}): Promise> { + const dir = options.configDir ?? getConfigDir(); + const s = stateFor(dir); + if (options.force) { + s.cooldown.clear(); + s.lastMissFetch.clear(); + } + const out = {} as Record; + await Promise.all(SOURCES.map(async def => { out[def.id] = await refreshSource(dir, s, def, { full: true, force: options.force }); })); + if (options.force) persist(dir, s); + return out; +} + +/** Called from GET /api/models and GET /api/usage: arms miss-driven fetching and refreshes a stale source. */ +export function triggerPriceCatalogRefresh(): void { + if (!autoAllowed()) return; + autoEnabled = true; + trackBackgroundWork("price-catalog-refresh", refreshPriceCatalogs().catch(() => undefined)); +} + +/** Test seam: inject the feed fetcher (also lifts the NODE_ENV=test no-network guard). */ +export function setPriceCatalogFetchForTests(impl: typeof fetch | null): void { + testFetch = impl; + autoEnabled = impl !== null; +} + +export function resetPriceCatalogForTests(): void { + states.clear(); + testFetch = null; + autoEnabled = false; +} diff --git a/src/usage/user-cost-overlays.ts b/src/usage/user-cost-overlays.ts index 058857858d9..7348c26a77a 100644 --- a/src/usage/user-cost-overlays.ts +++ b/src/usage/user-cost-overlays.ts @@ -25,6 +25,8 @@ export { COST4_RATE_KEYS, isValidCost4Rate, validCost4, isValidPromptLengthPrici import { redactSecretString } from "../lib/redact"; import { isSelectableCodexPoolAccount, MAIN_CODEX_ACCOUNT_ID } from "../codex/account-id"; import { codexAccountLogLabel } from "../codex/account-label"; +import { getProviderRegistryEntry } from "../providers/registry"; +import { priceCatalogVersion } from "./price-catalog-version"; const EMPTY: readonly ExpectedPriceOverlay[] = []; @@ -32,6 +34,8 @@ let active: readonly ExpectedPriceOverlay[] = EMPTY; let activeSignature = ""; let activeConfigured = new Set(); let activeAccountProviders = codexAccountProviders([]); +let activeMultipliers: ReadonlyMap = new Map(); +let activeSubscriptions: ReadonlySet = new Set(); let version = 0; let preservedDiskOnlyProviders: Record | null = null; @@ -78,6 +82,25 @@ function codexAccountProviders(accounts: OcxConfig["codexAccounts"]): Map 0, <= 1000. */ +export function isValidPriceMultiplier(value: unknown): value is number { + return typeof value === "number" && Number.isFinite(value) && value > 0 && value <= 1000; +} + +/** + * Providers whose requests are covered by a subscription, so a catalog price is only the + * API-equivalent: effective auth `oauth` (config `authMode`, else the registry) or `forward` + * (ChatGPT login), plus every exact Codex account identity. + */ +function subscriptionProviders(config: OcxConfig, accountProviders: ReadonlyMap): Set { + const out = new Set(accountProviders.keys()); + for (const [name, provider] of Object.entries(config.providers ?? {})) { + const auth = provider?.authMode ?? getProviderRegistryEntry(name)?.authKind; + if (auth === "oauth" || auth === "forward") out.add(name); + } + return out; +} + /** Register one active live-config owner. Multiple server leases may share one config object. */ export function registerPreservedProviderOwner(config: OcxConfig): void { const tagged = config as PreservationTaggedConfig; @@ -311,12 +334,23 @@ export function refreshUserCostOverlays(config: OcxConfig): void { const configuredNames = Object.keys(providers ?? {}).sort(); const accountProviders = codexAccountProviders(config.codexAccounts); const accountEntries = [...accountProviders].sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0); - const signature = `${JSON.stringify(configuredNames)}\u0000${JSON.stringify(rows)}\u0000${JSON.stringify(accountEntries)}`; + // Relay multipliers and the subscription set are part of the identity as well: both change a + // matched price after the memoized lookup, so a change must bump the version. + const multipliers = new Map(); + for (const [providerName, provider] of Object.entries(providers ?? {})) { + if (isValidPriceMultiplier(provider?.priceMultiplier)) multipliers.set(providerName, provider.priceMultiplier); + } + const subscriptions = subscriptionProviders(config, accountProviders); + const multiplierEntries = [...multipliers].sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0); + const subscriptionNames = [...subscriptions].sort(); + const signature = `${JSON.stringify(configuredNames)}\u0000${JSON.stringify(rows)}\u0000${JSON.stringify(accountEntries)}\u0000${JSON.stringify(multiplierEntries)}\u0000${JSON.stringify(subscriptionNames)}`; if (signature === activeSignature) return; activeSignature = signature; active = rows; activeConfigured = new Set(configuredNames); activeAccountProviders = accountProviders; + activeMultipliers = multipliers; + activeSubscriptions = subscriptions; version++; } @@ -325,9 +359,23 @@ export function activeUserCostOverlays(): readonly ExpectedPriceOverlay[] { return active; } -/** Monotonic version bumped on pricing-identity changes; used by the estimator memo key. */ +/** + * Monotonic version bumped on pricing-identity changes (overlay rows, provider set, multipliers, + * subscription set, and the online price catalog); used by the estimator memo key and the + * usage caches. Both counters only grow, so their sum changes whenever either does. + */ export function userCostOverlayVersion(): number { - return version; + return version + priceCatalogVersion(); +} + +/** Per-provider `priceMultiplier` from the last refresh, keyed by configured provider name. */ +export function activePriceMultipliers(): ReadonlyMap { + return activeMultipliers; +} + +/** Subscription-billed provider names and account identities from the last refresh. */ +export function activeSubscriptionProviders(): ReadonlySet { + return activeSubscriptions; } /** Configured provider names from the last refresh (pricing-namespace identity). */ diff --git a/structure/INDEX.md b/structure/INDEX.md index 2bea8d6e528..cb9e6cf3448 100644 --- a/structure/INDEX.md +++ b/structure/INDEX.md @@ -134,7 +134,7 @@ The dashboard, the management API, and third-party client config ownership. | [`dashboard-and-usage.md`](dashboard-and-usage.md) | Dashboard page contracts, usage accounting and request metrics, and per-surface management settings. | | [`usage-client-attribution.md`](usage-client-attribution.md) | Privacy-bounded client labels carried from request ingress into usage records. | | [`quota-pool.md`](quota-pool.md) | The passive GET /api/quota-pool read GOAT consumes (ocx.quota-pool/1): sources, window mapping, absent-not-zero, and the install identity behind homeId and accountRef. | -| [`pricing.md`](pricing.md) | User price policies, prompt-length rate selection, persistence and estimate cache invalidation. | +| [`pricing.md`](pricing.md) | User price policies, prompt-length rate selection, automatic price catalog tiers, relay multiplier, persistence and estimate cache invalidation. | | [`clients/export-model-names.md`](clients/export-model-names.md) | Per-connection naming policy, router-proven aliases and passive export previews. | | [`clients/omo-export-dialect.md`](clients/omo-export-dialect.md) | Opt-in Responses dialect for the managed Pi-based omo export and its refresh lifecycle. | | [`clients/integrations.md`](clients/integrations.md) | Third-party client config ownership, snapshots, refresh, disable, and restore. | @@ -479,6 +479,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/server/management/link-routes.ts` | [`remote-link.md`](remote-link.md) | | `src/server/management/logs-usage-routes.ts` | [`usage-client-attribution.md`](usage-client-attribution.md) | | `src/server/management/model-routes.ts` | [`pricing.md`](pricing.md) | +| `src/server/management/model-row-price.ts` | [`pricing.md`](pricing.md) | | `src/server/management/provider-overwrite-carry.ts` | [`providers/hosted-tool-denials.md`](providers/hosted-tool-denials.md) | | `src/server/management/quota-pool-routes.ts` | [`quota-pool.md`](quota-pool.md) | | `src/server/management/telemetry-routes.ts` | [`telemetry.md`](telemetry.md) | @@ -579,6 +580,8 @@ A source area can be described by more than one doc, because these docs are orga | `src/usage/cost.ts` | [`pricing.md`](pricing.md) | | `src/usage/error-detail.ts` | [`data-planes/error-detail.md`](data-planes/error-detail.md) | | `src/usage/link-log-fields.ts` | [`remote-link.md`](remote-link.md) | +| `src/usage/price-catalog-version.ts` | [`pricing.md`](pricing.md) | +| `src/usage/price-catalog.ts` | [`pricing.md`](pricing.md) | | `src/usage/source-filter.ts` | [`usage-client-attribution.md`](usage-client-attribution.md) | | `src/usage/summary.ts` | [`usage-client-attribution.md`](usage-client-attribution.md) | | `src/usage/timeline.ts` | [`gui-and-management-api.md`](gui-and-management-api.md) | diff --git a/structure/gui-and-management-api.md b/structure/gui-and-management-api.md index d5dc67e0b9d..e130d897477 100644 --- a/structure/gui-and-management-api.md +++ b/structure/gui-and-management-api.md @@ -290,7 +290,7 @@ per-request first-party callback reads that live object; a failed write leaves i | Diagnostics/sync | `src/server/management/config-routes.ts` — `GET /api/diagnostics/project-config` reports project-level Codex config that bypasses managed routing; `POST /api/sync` re-runs catalog/config sync. The diagnostic reports the bypass; it does not rewrite the project file. | | Sidecar/shadow-call settings | `src/server/management/config-routes.ts` — `GET/PUT /api/sidecar-settings` and `GET/PUT /api/shadow-call-settings`, and `GET/PUT /api/auto-review-settings`. The auto-review settings route validates the global reviewer selector, persists its enabled/model fields with rollback before publication, then converges the catalog and returns its disposition; the Models panel keeps the server echo and displays a pending-sync notice whenever no commit is reported. Sidecar PUT accepts model and backend (web-search union: openai/anthropic/xai/gemini/exa; xAI is live through stored Grok OAuth, while Gemini/Exa remain inert until their executors ship) plus validated `webSearch.xSearch`, optional `webSearch.exaApiKey` (write/clear only — never echoed by GET or the PUT response; redact.ts strips it from logs), `webSearch.reasoning`, `vision.reasoning`, `vision.enabled`, `vision.maxDescriptionsPerTurn`, and `vision.timeoutMs`; the read and PUT-response payload reports model, backend, reasoning, enabled, the vision per-turn limit, and timeout. An effective `vision.enabled` change persists before one bounded catalog-only convergence updates the managed catalog and Codex model cache; the PUT response returns `catalogRefreshPending` for incomplete work. Identical effective values and unrelated partial writes do not converge, and no result claims client reload. `timeoutMs` is validated against the runtime integer bounds in `src/vision/timeout-bounds.ts`. Provider/OAuth credentials live in their stores; `exaApiKey` is the one sidecar-owned secret and follows the write-only contract above. Both shadow-call responses also report the resolved `sourceModels` — the prefixes the runtime actually intercepts (`src/lib/shadow-call.ts`, default `gpt-6-luna` and `gpt-5.6-luna`; the retired `gpt-5.4-mini` stays available as an explicit `sourceModels` entry for 0.144.x clients), so no client hard-codes a helper slug that a Codex release can invalidate. PUT refuses a qualified target that only the router's default-provider fallback accepts. Provider disable and delete report the target they leave behind as `dependentShadowIntercept` (`shadowInterceptProviderDependency` in `src/server/management/shadow-call-validation.ts`), and at request time an unresolvable target returns `409 intercept_target_unavailable` before any send (`src/server/responses/shadow-target-availability.ts`); it never falls back to the native source model or the default provider. | | Storage | `src/server/management/logs-usage-routes.ts` — `GET /api/storage`, `POST /api/storage/cleanup/preview` and `/api/storage/cleanup`, `GET /api/storage/trash`, `POST /api/storage/trash/restore`, and `GET/PUT /api/storage/cleanup-policy` plus `POST /api/storage/cleanup-policy/run`. `GET /api/storage/cleanup-policy/test-stream` and `GET /api/storage/trash/restore/test-stream` exist for progress-stream testing. Cleanup takes an explicit `mode`: `quarantine` moves to trash and is restorable, `permanent` is not. The caller must name the mode — there is no default that silently deletes. | -| Provider quotas and tests | `src/server/management/provider-routes.ts` — `GET /api/provider-quotas`, `POST /api/providers/test`, `POST /api/providers/test-completion` (explicit single-model completion diagnostic, [contract](provider-completion-diagnostic.md)), `GET/PUT /api/provider-context-caps`, `GET /api/provider-presets`. A quota read may be served from cache or force-refreshed; absent quota data is reported as unknown rather than as a measured zero. `GET /api/quota-pool` (`src/server/management/quota-pool-routes.ts`, loaded on demand) is the always-passive `ocx.quota-pool/1` read GOAT consumes; see [quota pool](quota-pool.md). | +| Provider quotas and tests | `src/server/management/provider-routes.ts` — `GET /api/provider-quotas`, `POST /api/providers/test`, `POST /api/providers/test-completion` (explicit single-model completion diagnostic, [contract](provider-completion-diagnostic.md)), `POST /api/providers/probe-capabilities` (`src/server/management/provider-capability-probe.ts`, loaded on demand: the same diagnostic with an image or reasoning-effort body variant, sequential, metered providers need `confirmMetered`, results are codes and statuses only; `apply` writes conclusive results into `modelCapabilities`/`modelReasoningEfforts` and labels them in `modelCapabilityProbes`, which a manual PATCH or `PUT /api/model-settings` of that axis removes), `GET/PUT /api/provider-context-caps`, `GET /api/provider-presets`. A quota read may be served from cache or force-refreshed; absent quota data is reported as unknown rather than as a measured zero. `GET /api/quota-pool` (`src/server/management/quota-pool-routes.ts`, loaded on demand) is the always-passive `ocx.quota-pool/1` read GOAT consumes; see [quota pool](quota-pool.md). | | Models and visibility | `src/server/management/model-routes.ts` — `GET /api/models`, `PUT /api/disabled-models`, `PUT /api/model-visibility`, `PUT /api/selected-models`, `GET/POST /api/custom-models`, `PUT /api/model-settings`. Visibility writes trigger catalog sync through the owning server path. `PUT /api/model-settings` edits one routed row's capability axes in place — context window, declared input modalities, reasoning ladder and its default — through the provider-level per-model maps the runtime already reads, so the row keeps its discovery provenance instead of being replaced by a custom model. Every field is optional and `null` clears the declaration back to the registry/catalog answer; display name is deliberately absent because `PUT /api/providers/{provider}/model-display-names` already owns it. | | Effort and fallback | `src/server/management/agent-settings-routes.ts` — `GET/PUT /api/effort-caps`, `/api/subagent-models`, `/api/subagent-model-fallback`. Caps clamp; they do not reject. | | Grok and Claude integrations | `src/server/management/agent-settings-routes.ts` — `GET /api/grok`, `PUT /api/grok/selection`, `POST /api/grok/apply`, `GET/PUT /api/claude-desktop`, `POST /api/claude-desktop/apply` (`mode`: `gateway` default for new installs, `first-party` opt-in, or legacy shapes), `GET /api/claude-desktop/status` (`mode`, `riskWarning`, `firstParty`), `GET/PUT /api/claude-code`. Gateway apply writes an external app's profile, so its status probe must read the same resolved path it writes (see [`responses.md`](transports/responses.md)); first-party apply writes only the Claude Code proxy env, see [`clients/claude-desktop.md`](clients/claude-desktop.md#desktop-modes-gateway-and-first-party). `gui/src/pages/ClaudeDesktop.tsx` renders the mode selector and sends the chosen `mode` with apply. It leads with Default model and Quick task model, which set the `opus` and `haiku` family defaults through `gui/src/pages/claude-desktop-roles.ts` (same move rules as the family lanes, which sit under a folded Advanced disclosure); the profile wire shape is unchanged. `gui/src/pages/ClaudeCode.tsx` is one page with a single Save bar; its Save body (`gui/src/pages/claude-code-save.ts`) omits the top-level `enabled`, which only the immediate connection switch writes. `PUT /api/claude-code` re-runs macOS system-env reconciliation (`src/server/system-env.ts`) whenever the body carries `systemEnv`, `authMode`, a model slot or a lever field: keys opencodex tracks as injected are refreshed or unset once the config stops producing them, and a launchd value the user set before injection is never touched. `gui/src/pages/claude-code-tier-defaults.tsx` edits `tierModels` through the Save bar's dirty/poll/revert contract. `GET /api/claude-code/model-defaults` previews local user defaults; `POST` accepts exactly a fingerprint bound to the full settings document and saved tiers (`src/claude/model-default-settings.ts`). Changed settings/tiers return 409 and existing empty/corrupt documents are refused. Apply stages private bytes, captures the actual shared file into `model-default-backups/apply-*/settings.json` under the Claude config directory, validates that capture, then publishes with an exclusive hard link. A competing destination is never replaced, even on rollback; there is a brief missing-file interval, and unsupported cross-volume/link operations have no overwrite fallback. Failed restoration returns `settings_recovery_required` with the private recovery path shown as an alert. Selected aliases receive paired `_MODEL_NAME` labels via `claudeTierDisplayName` in `src/claude/context-windows.ts`. Preview includes label additions/updates and uses an empty `after` string for removal. An existing label is managed only when it exactly matches the generated name for its current alias (an identical manually entered label is indistinguishable); custom, empty and non-string labels are preserved. Unset tiers retain both their settings model IDs and display labels; a matching generated label is removed only when an explicitly selected tier is replaced with a non-alias; no client restart, connection credential change or Desktop family edit occurs. Covered by `tests/claude-integration/claude-model-default-labels.test.ts`, `tests/claude-integration/claude-intercept-settings.test.ts`, `tests/server/model-settings-management-api.test.ts`, `gui/tests/claude-code-save.test.ts` and `gui/tests/claude-code-background-helper.test.tsx`. Claude sidecar overrides have their own optional boolean `enabled` in GET and validated PUT. `gui/src/pages/claude-code-sidecar.ts` preserves an untouched off override even without a backend/model; Inherit explicitly clears it, while selecting a nonempty model can re-enable it through the existing target validation. Model edits preserve an already enabled override; typing then clearing a model in an off draft returns it to off. Draft provenance never enters PUT or confirmed saved state. | @@ -301,7 +301,7 @@ through `commitProviderPatch`; an unpublished save failure restores the live gra write clears the provider model cache. Its receipt separates `changed`, `saved`, `hasOverrides`, and `catalogRefresh` (committed/skipped/failed), so a no-op does not imply the absence of stored overrides. Routed `GET /api/models` rows expose the exact `contextWindowDeclared` alongside the -effective `contextWindow`. The dashboard dialog seeds from that declaration, shows the inherited +effective `contextWindow`, plus per-axis `capabilitySources` (manual > probed > reported > inferred > unknown) and display-only same-name `inferredCapabilities` from `src/server/management/model-capability-sources.ts` over `src/providers/model-capability-inference.ts` (built-in original-vendor registry, bundled vendor metadata, other configured providers' declarations, bundled aggregators; a disagreeing tier answers nothing and does not fall through). Estimates never route and never enter the catalog; `tests/providers/model-capability-inference.test.ts` pins precedence and labels. The dashboard dialog seeds from that declaration, shows the inherited effective window when absent, and offers a read-only Reload after an unknown save outcome or a confirmed save whose list reload failed; a failed or retryable Codex catalog refresh is a warning after the save, and a non-retryable skip (no managed catalog) is a clean save. Close returns focus to the opening row control. diff --git a/structure/manifest.json b/structure/manifest.json index efe4a5bbca4..b6812594754 100644 --- a/structure/manifest.json +++ b/structure/manifest.json @@ -1059,7 +1059,7 @@ "path": "pricing.md", "tier": 5, "title": "Model Price Overrides", - "scope": "User price policies, prompt-length rate selection, persistence and estimate cache invalidation.", + "scope": "User price policies, prompt-length rate selection, automatic price catalog tiers, relay multiplier, persistence and estimate cache invalidation.", "documents": [ "src/config/live-reconcile.ts", "src/cli/models-runtime.ts", @@ -1069,6 +1069,9 @@ "src/usage/cost.ts", "src/usage/user-cost-overlays.ts", "src/usage/user-cost-validation.ts", + "src/usage/price-catalog.ts", + "src/usage/price-catalog-version.ts", + "src/server/management/model-row-price.ts", "gui/src/components/ModelPriceDialog.tsx" ] }, diff --git a/structure/pricing.md b/structure/pricing.md index 7d9cf8d0ec7..ca25ac79e80 100644 --- a/structure/pricing.md +++ b/structure/pricing.md @@ -64,3 +64,62 @@ policy-only edits invalidate price memoization and usage caches; identical reloa These settings affect estimates, not upstream billing or official catalog price lists. Only reported or estimated usage is priced by `src/usage/cost.ts`. Partial tokens on an unreported or unsupported attempt do not establish its full cost. `src/usage/summary.ts` marks a multi-attempt request unpriced when any potentially billed attempt lacks measured usage, while a refusal before generation without usage remains a known zero. Measured attempts retain their individual estimates. + +## Automatic prices and sources + +Price resolution in `src/usage/cost.ts` runs in a fixed order and `MatchedPrice.source` records the +winner: `user` (`providers..modelCosts`, shown as manual) > `expected` or `jawcode` (verified +official override, bundled jawcode bundle, expected overlay, jawcode model-level vendor price, shown +as official) > `models-dev` > `openrouter` > unknown. The last two tiers run only where the +model-level fallback would otherwise return null, and only when the caller leaves +`allowModelLevelFallback` on. A catalog price has `status: "verified-derived"`, so `estimated` is +true; `sourceRef` is the feed URL and `verifiedAt` the fetch time. A user price, including explicit +all-zero rates, is never replaced or scaled. + +`src/usage/price-catalog.ts` owns the two public lists. A model matches a row by +`priceCatalogKey`: the `cn:` channel prefix, any `vendor/` or `zd/` prefix, case, dots versus +hyphens and a `[...]` suffix are ignored. Within one source, two rows with the same key and +different prices make the key ambiguous and store nothing for it. models.dev rows count only for the +model's own vendor providers (resellers are ignored); its rates are already per 1M tokens. OpenRouter +rows come from `data[]`, skip ids that contain `:` (such as `:free`) or start with `openrouter/`, +and multiply the per-token strings by 1,000,000. Rows with both input and output zero, or any invalid +rate, are skipped. Lookup is synchronous from `/price-catalog-cache.json` (loaded lazily, +a missing or corrupt file means no catalog) and never touches the network. +`src/usage/price-catalog-version.ts` holds the version counter that +`userCostOverlayVersion()` folds in, so the estimate memo and the usage caches rebuild when a +snapshot changes. + +Refresh rules, all in the background and never on a request path: + +- Daily: `refreshPriceCatalogs` replaces a source older than 24 hours wholesale. Concurrent callers + share one fetch per source. A failed source keeps its previous snapshot and is not retried for 60 + seconds; the status carries a short error code and never a body. Fetches use `redirect: "error"`, + no credentials, a 15 second timeout and a 32 MiB cap. +- Miss-driven: a model no tier could price is queued by `notePriceCatalogMiss`. All misses in one + tick share one fetch per source (the feeds are whole-table GETs), and only the queued keys are + merged into the existing snapshot, adding missing keys and leaving every other row alone. A key + still unfound afterwards cools down for 6 hours (persisted in the cache file) and a source gets at + most one miss-driven fetch per 6 hour window. The resolver returns null at once and the next + lookup after the fetch finds the price. +- `GET /api/models` and `GET /api/usage` arm the miss path and start the daily refresh; + `POST /api/pricing/refresh` forces both sources and clears cooldowns. Under `NODE_ENV=test` nothing + fetches unless a test injects a fetcher with `setPriceCatalogFetchForTests`. + +`providers..priceMultiplier` (finite, greater than 0, at most 1000; absent means 1, stored +sparse) scales every non-user price for that provider's configured name. It is validated by +`priceMultiplierConfigError`, dropped with a warning by `sanitizeModelCostsForLoad`, and set with +`PUT /api/providers/{provider}/price-multiplier` (`{ multiplier: number | null }`; `null` deletes the +key; the write rolls back when persistence fails). `GET /api/providers/{provider}/model-costs` returns +`priceMultiplier` when set. A provider is a subscription when its effective auth (`authMode`, else +the registry) is `oauth` or `forward`, or it is an exact Codex account identity; its non-user prices +carry `apiEquivalent`. `refreshUserCostOverlays` collects the multipliers and the subscription set +and includes both in the change signature, so editing either bumps the version. +`src/server/management/model-row-price.ts` projects the effective price onto each non-combo +`/api/models` row as `price` (`manual`, `official`, `models-dev` or `openrouter`; absent when +unknown). `costResult` adds `online_price_catalog`, `relay_price_multiplier` and +`subscription_api_equivalent` to the estimate reasons. + +Tests: `tests/usage/price-catalog.test.ts` (matching, parsing, cache), +`tests/usage/price-catalog-miss.test.ts` (miss, cooldown, merge), +`tests/usage/price-catalog-resolution.test.ts` (tiers, multiplier, subscription, reasons) and +`tests/server/price-catalog-management-api.test.ts` (both routes, load, row `price`). diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index ebfabf37355..5825bb94174 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -172,6 +172,9 @@ "integrations-writer-guard.test.ts": "clients", "aside-profiles-catalog-refresh.test.ts": "clients", "azure-vendor-metadata.test.ts": "providers", + "model-capability-inference.test.ts": "providers", + "provider-capability-probe.test.ts": "providers", + "provider-capability-probe-provenance.test.ts": "server", "anthropic-instance-isolation.test.ts": "adapters/anthropic", "anthropic-instance-pool-parity.test.ts": "adapters/anthropic", "anthropic-instance-quota.test.ts": "adapters/anthropic", "anthropic-instance-recovery.test.ts": "adapters/anthropic", "anthropic2-fast-parity.test.ts": "adapters/anthropic", "anthropic2-native-routing.test.ts": "claude-integration", @@ -1085,7 +1088,7 @@ "mimo-v26-catalog.test.ts": "providers", "minimax-clients.test.ts": "providers", "minimax-quota.test.ts": "providers", "minimax-reasoning-split.test.ts": "providers", "model-cache-generation-tombstone.test.ts": "codex-integration", "model-cache.test.ts": "codex-integration", - "model-costs-management-api.test.ts": "server", "model-discovery-management-api.test.ts": "server", + "model-costs-management-api.test.ts": "server", "price-catalog-management-api.test.ts": "server", "model-discovery-management-api.test.ts": "server", "server-new-model-policy-arrival.test.ts": "server", "model-display-names-management-api.test.ts": "codex-integration", "model-metadata-sync.test.ts": "codex-integration", "model-pinned-effort-config.test.ts": "config", "model-pinned-effort.test.ts": "codex-integration", @@ -1510,7 +1513,7 @@ "upstream-retry-zero-output.test.ts": "lib", "upstream-transient-retry.test.ts": "providers", "url-normalization.test.ts": "config", "usage-aggregate-cache.test.ts": "usage", "usage-anthropic-fast-pricing.test.ts": "usage", "usage-cursor-fast-pricing.test.ts": "usage", - "usage-attempt-delivery.test.ts": "usage", "usage-client-attribution.test.ts": "usage", "usage-cost.test.ts": "usage", "usage-haiku-5-5-pricing.test.ts": "usage", "usage-fable-5-2-pricing.test.ts": "usage", "gpt61-terra-pricing.test.ts": "usage", + "usage-attempt-delivery.test.ts": "usage", "usage-client-attribution.test.ts": "usage", "usage-cost.test.ts": "usage", "usage-haiku-5-5-pricing.test.ts": "usage", "usage-fable-5-2-pricing.test.ts": "usage", "gpt61-terra-pricing.test.ts": "usage", "price-catalog.test.ts": "usage", "price-catalog-miss.test.ts": "usage", "price-catalog-resolution.test.ts": "usage", "usage-debug.test.ts": "usage", "usage-failure-fingerprint.test.ts": "usage", "usage-filter-index.test.ts": "usage", "usage-filter-paging.test.ts": "usage", "usage-failure-persistence.test.ts": "usage", "usage-failure-projection.test.ts": "usage", diff --git a/tests/helpers/price-catalog-fixtures.ts b/tests/helpers/price-catalog-fixtures.ts new file mode 100644 index 00000000000..8a14eefe79b --- /dev/null +++ b/tests/helpers/price-catalog-fixtures.ts @@ -0,0 +1,124 @@ +import { afterEach, beforeEach, setSystemTime } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { resetPriceCatalogForTests, setPriceCatalogFetchForTests } from "../../src/usage/price-catalog"; +import { refreshUserCostOverlays } from "../../src/usage/user-cost-overlays"; +import { removeTreeWithRetry } from "./remove-tree"; + +export const MODELS_DEV_URL = "https://models.dev/api.json"; +export const OPENROUTER_URL = "https://openrouter.ai/api/v1/models"; + +/** Hand-written miniature of the models.dev feed: fake ids, one row per rule under test. */ +export function modelsDevFeed(): Record { + return { + deepseek: { models: { + "zephyr-flash-2": { cost: { input: 0.1, output: 0.4, cache_read: 0.01, cache_write: 0.2 } }, + "twin-1": { cost: { input: 1, output: 2 } }, + "same-1": { cost: { input: 5, output: 6 } }, + } }, + moonshotai: { models: { "moon-pro-3.1": { cost: { input: 1.5, output: 6 } } } }, + xai: { models: { "Grok-Quill-5.5": { cost: { input: 2, output: 8, cache_read: 0.5 } } } }, + google: { models: { + "gem-free-4": { cost: { input: 0, output: 0 } }, + "gem-nocost-4": { name: "no cost block" }, + "gem-badrate-4": { cost: { input: -1, output: 2 } }, + } }, + alibaba: { models: { + "acme/twin-1": { cost: { input: 3, output: 4 } }, + "acme/same-1": { cost: { input: 5, output: 6 } }, + } }, + resellerhub: { models: { + "zephyr-flash-2": { cost: { input: 9, output: 9 } }, + "reseller-only-1": { cost: { input: 7, output: 7 } }, + } }, + }; +} + +/** Hand-written miniature of the OpenRouter feed (USD per token, as strings). */ +export function openRouterFeed(): Record { + return { data: [ + { id: "deepseek/zephyr-flash-2", pricing: { prompt: "0.0000003", completion: "0.0000012" } }, + { id: "acme/orb-only-7", pricing: { prompt: "0.0000005", completion: "0.000002", input_cache_read: "0.0000001", input_cache_write: "0.00000125" } }, + { id: "deepseek/zephyr-flash-2:free", pricing: { prompt: "0", completion: "0" } }, + { id: "acme/free-only-1:free", pricing: { prompt: "0.000001", completion: "0.000002" } }, + { id: "openrouter/auto", pricing: { prompt: "0.000001", completion: "0.000002" } }, + { id: "acme/negative-1", pricing: { prompt: "-0.000001", completion: "-0.000002" } }, + { id: "acme/dup-2", pricing: { prompt: "0.000001", completion: "0.000002" } }, + { id: "other/dup-2", pricing: { prompt: "0.000009", completion: "0.000002" } }, + ] }; +} + +export interface FeedFetch { + readonly fetch: typeof fetch; + /** Every requested URL, in order. */ + readonly calls: string[]; + callsTo(url: string): number; + /** URLs that fail with a network error. */ + readonly failing: Set; + /** Replace the body served for a URL. */ + serve(url: string, body: unknown): void; + /** Hold every response until the returned function is called. */ + hold(): () => void; +} + +export function feedFetch(): FeedFetch { + const bodies = new Map([[MODELS_DEV_URL, modelsDevFeed()], [OPENROUTER_URL, openRouterFeed()]]); + const calls: string[] = []; + const failing = new Set(); + let gate: Promise | null = null; + const impl = async (input: string | URL | Request): Promise => { + const url = typeof input === "string" ? input : input instanceof URL ? input.href : input.url; + calls.push(url); + if (gate) await gate; + if (failing.has(url)) throw new TypeError("network down"); + if (!bodies.has(url)) return new Response("not found", { status: 404 }); + const body = bodies.get(url); + return new Response(typeof body === "string" ? body : JSON.stringify(body), { status: 200, headers: { "content-type": "application/json" } }); + }; + return { + fetch: impl as unknown as typeof fetch, + calls, + callsTo: url => calls.filter(call => call === url).length, + failing, + serve: (url, body) => { bodies.set(url, body); }, + hold: () => { + let release!: () => void; + gate = new Promise(resolve => { release = resolve; }); + return () => { release(); gate = null; }; + }, + }; +} + +/** + * Per-test sandbox: a private config dir for the catalog cache file, a clean catalog state and + * no leftover user overlays, multipliers or frozen clock. Registers its own hooks. + */ +export function installCatalogSandbox(): { readonly dir: string; readonly feeds: FeedFetch } { + let home = ""; + let previousHome: string | undefined; + let feeds = feedFetch(); + beforeEach(() => { + previousHome = process.env["OPENCODEX_HOME"]; + home = mkdtempSync(join(tmpdir(), "ocx-price-catalog-")); + process.env["OPENCODEX_HOME"] = home; + resetPriceCatalogForTests(); + feeds = feedFetch(); + setPriceCatalogFetchForTests(feeds.fetch); + }); + afterEach(() => { + setSystemTime(); + resetPriceCatalogForTests(); + refreshUserCostOverlays({ port: 0, defaultProvider: "none", providers: {} }); + if (previousHome === undefined) delete process.env["OPENCODEX_HOME"]; + else process.env["OPENCODEX_HOME"] = previousHome; + removeTreeWithRetry(home); + }); + return { + get dir() { return home; }, + get feeds() { return feeds; }, + }; +} + +/** One macrotask: lets a `setTimeout(0)` miss flush start, then the caller settles background work. */ +export const nextTick = (): Promise => new Promise(resolve => setTimeout(resolve, 5)); diff --git a/tests/providers/model-capability-inference.test.ts b/tests/providers/model-capability-inference.test.ts new file mode 100644 index 00000000000..ea34e2a404f --- /dev/null +++ b/tests/providers/model-capability-inference.test.ts @@ -0,0 +1,76 @@ +import { describe, expect, test } from "bun:test"; +import { capabilityInferenceKey, createModelCapabilityInference } from "../../src/providers/model-capability-inference"; +import { modelCapabilitySourceResolver } from "../../src/server/management/model-capability-sources"; +import type { OcxConfig, OcxProviderConfig } from "../../src/types"; + +const relay = (extra: Partial = {}): OcxProviderConfig => ({ + adapter: "openai-chat", baseUrl: "https://relay.example/v1", apiKey: "k", models: ["claude-opus-5-5"], liveModels: false, ...extra, +}); + +describe("same-name capability inference", () => { + test("normalizes vendor prefixes, case, dots and bracket suffixes", () => { + expect(capabilityInferenceKey("Anthropic/Claude-Opus-5.5[1m]")).toBe("claude-opus-5-5"); + }); + + test("a relay's Claude model infers all three axes from the built-in Anthropic registry", () => { + const inferred = createModelCapabilityInference({ port: 0, providers: { zayu: relay() } } as OcxConfig)("claude-opus-5-5", "zayu"); + expect(inferred.contextWindow).toEqual({ value: 1_000_000, basis: { source: "registry", provider: "anthropic", modelId: "claude-opus-5-5" } }); + expect(inferred.inputModalities?.value).toContain("image"); + expect(inferred.inputModalities?.basis.source).toBe("registry"); + expect(inferred.reasoningEfforts?.value).toEqual(expect.arrayContaining(["low", "high"])); + }); + + test("another configured provider's declaration answers when no bundled source knows the model", () => { + const config = { + port: 0, + providers: { + zayu: relay({ models: ["house-model-7"] }), + other: relay({ models: ["house-model-7"], modelContextWindows: { "house-model-7": 64_000 }, modelReasoningEfforts: { "house-model-7": ["low", "high"] } }), + }, + } as OcxConfig; + const inferred = createModelCapabilityInference(config)("house-model-7", "zayu"); + expect(inferred.contextWindow).toEqual({ value: 64_000, basis: { source: "provider", provider: "other", modelId: "house-model-7" } }); + expect(inferred.reasoningEfforts?.value).toEqual(["low", "high"]); + // The provider itself is excluded: its own declaration is manual, not an inference. + expect(createModelCapabilityInference(config)("house-model-7", "other").contextWindow).toBeUndefined(); + }); + + test("conflicting declarations in one tier answer nothing instead of picking one", () => { + const config = { + port: 0, + providers: { + a: relay({ models: ["house-model-8"], modelContextWindows: { "house-model-8": 64_000 } }), + b: relay({ models: ["house-model-8"], modelContextWindows: { "house-model-8": 128_000 } }), + }, + } as OcxConfig; + expect(createModelCapabilityInference(config)("house-model-8", "zayu").contextWindow).toBeUndefined(); + }); + + test("an unknown model infers nothing", () => { + expect(createModelCapabilityInference()("no-such-model-xyz")).toEqual({}); + }); +}); + +describe("capability sources on /api/models rows", () => { + const config = (extra: Partial = {}) => ({ port: 0, providers: { zayu: relay(extra) } } as OcxConfig); + const row = { provider: "zayu", id: "claude-opus-5-5" }; + + test("an id-only relay row is unknown on every axis and carries the same-name estimates", () => { + const fields = modelCapabilitySourceResolver(config())(row, {}); + expect(fields.capabilitySources).toEqual({ contextWindow: "unknown", inputModalities: "unknown", reasoningEfforts: "unknown" }); + expect(fields.inferredCapabilities?.contextWindow?.value).toBe(1_000_000); + }); + + test("manual beats reported beats inferred, and a probe-written declaration is labeled probed", () => { + const resolve = modelCapabilitySourceResolver(config({ + modelCapabilityProbes: { "claude-opus-5-5": { probedAt: "2026-11-13T00:00:00.000Z", inputModalities: true } }, + })); + const fields = resolve( + { ...row, contextWindow: 200_000, inputModalities: ["text", "image"] }, + { contextWindowDeclared: 200_000, inputModalitiesDeclared: ["text", "image"], reasoningEfforts: ["low"] }, + ); + expect(fields.capabilitySources).toEqual({ contextWindow: "manual", inputModalities: "probed", reasoningEfforts: "reported" }); + const inferredRow = resolve({ ...row, contextWindow: 1_000_000, contextWindowInference: { source: "vendor", provider: "anthropic", modelId: "claude-opus-5-5", contextWindow: 1_000_000 } }, {}); + expect(inferredRow.capabilitySources.contextWindow).toBe("inferred"); + }); +}); diff --git a/tests/providers/provider-capability-probe.test.ts b/tests/providers/provider-capability-probe.test.ts new file mode 100644 index 00000000000..61477b11c35 --- /dev/null +++ b/tests/providers/provider-capability-probe.test.ts @@ -0,0 +1,188 @@ +import { describe, expect, test } from "bun:test"; +import { chat, fixture, frame } from "../helpers/provider-completion-fixture"; +import { handleProviderCapabilityProbeRoute } from "../../src/server/management/provider-capability-probe"; +import { COMPLETION_TEST_IMAGE_DATA_URL, COMPLETION_TEST_PROMPT } from "../../src/server/management/provider-completion-test"; +import type { OcxConfig, OcxProviderConfig } from "../../src/types"; + +type Respond = (body: Record, init?: RequestInit) => Response | Promise; +const ok = () => new Response(chat()); +const status = (code: number) => new Response(JSON.stringify({ error: { message: "upstream detail sk-fixture" } }), { status: code }); +const isImage = (body: Record) => Array.isArray(body.messages?.[0]?.content) || Array.isArray(body.input); +const effortOf = (body: Record): string | undefined => body.reasoning_effort ?? body.reasoning?.effort; + +const isBaseline = (body: Record) => !isImage(body) && effortOf(body) === undefined; +function probeFixture(respond: Respond, overrides: Partial = {}, baselineRespond: Respond = ok) { + const f = fixture({ models: ["test-model"], ...overrides }, init => { + const body = JSON.parse(String(init?.body)); + return isBaseline(body) ? baselineRespond(body, init) : respond(body, init); + }); + const saves: OcxConfig[] = []; + let convergences = 0; + const probe = async (body: unknown, probeDeps = {}) => { + const req = new Request("http://127.0.0.1/api/providers/probe-capabilities", { + method: "POST", headers: { Host: "127.0.0.1", "content-type": "application/json" }, body: JSON.stringify(body), + }); + const response = await handleProviderCapabilityProbeRoute({ + req, config: f.config, + deps: { saveConfigPreservingClaudeCode: config => { saves.push(JSON.parse(JSON.stringify(config))); } }, + convergeCodexCatalog: async () => { convergences++; return { status: "committed", changed: false, degraded: false, notices: [] }; }, + }, probeDeps); + return { status: response.status, body: await response.json() as any }; + }; + return { ...f, saves, probe, convergences: () => convergences }; +} +// The first call is always the plain baseline prompt. +const bodies = (calls: { init?: RequestInit }[]) => calls.slice(1).map(call => JSON.parse(String(call.init!.body))); + +describe("capability probe: gates", () => { + test("metered provider without confirmation sends nothing and reports the plan", async () => { + const f = probeFixture(ok); + expect(await f.probe({ name: "fixture", model: "test-model", efforts: ["low", "high"] })).toEqual({ status: 200, + body: { applicable: true, sent: false, requiresConfirmation: true, metered: true, plannedRequests: 4 } }); + expect(f.calls).toHaveLength(0); + }); + test("local destination is not metered and needs no confirmation", async () => { + const f = probeFixture(ok, { baseUrl: "http://127.0.0.1:11434/v1", allowPrivateNetwork: true }); + const result = await f.probe({ name: "fixture", model: "test-model", checks: ["image"] }); + expect(result.body).toMatchObject({ applicable: true, sent: true, metered: false, baseline: { status: "accepted" }, image: { status: "accepted" } }); + expect(f.calls).toHaveLength(2); + }); + for (const [overrides, reason] of [[{ authMode: "oauth" }, "oauth_auth"], [{ authMode: "forward" }, "forward_auth"], + [{ disabled: true }, "disabled"], [{ adapter: "anthropic" }, "unsupported_adapter"]] as const) { + test(`${reason} is not applicable and sends nothing`, async () => { + const f = probeFixture(ok, overrides as Partial); + const result = await f.probe({ name: "fixture", model: "test-model", confirmMetered: true }); + expect(result.body).toMatchObject({ applicable: false, reason, sent: false }); + expect(f.calls).toHaveLength(0); + }); + } + const invalid: unknown[] = [null, [], {}, { name: "fixture" }, { model: "test-model" }, { name: "fixture", model: " " }, + { name: "fixture", model: "a\nb" }, { name: "fixture", model: "x".repeat(257) }, { name: "fixture", model: "m", extra: 1 }, + { name: "fixture", model: "m", checks: [] }, { name: "fixture", model: "m", checks: ["video"] }, + { name: "fixture", model: "m", efforts: ["turbo"] }, { name: "fixture", model: "m", efforts: [] }, + { name: "fixture", model: "m", efforts: ["none", "minimal", "low", "medium", "high", "xhigh", "max"] }, + { name: "fixture", model: "m", checks: ["image"], efforts: ["low"] }, { name: "fixture", model: "m", apply: "yes" }, + { name: "fixture", model: "m", confirmMetered: 1 }, { name: "fixture", model: "__proto__", apply: true }]; + for (const [index, body] of invalid.entries()) test(`invalid body ${index}: 400 and no send`, async () => { + const f = probeFixture(ok); + expect((await f.probe(body)).status).toBe(400); + expect(f.calls).toHaveLength(0); + }); + test("unknown provider is 404", async () => { + const f = probeFixture(ok); + expect(await f.probe({ name: "absent", model: "m" })).toEqual({ status: 404, body: { error: "unknown provider" } }); + }); +}); + +describe("capability probe: wire variants and classification", () => { + test("chat image variant and reasoning_effort ride the completion-test body", async () => { + const f = probeFixture(body => isImage(body) ? ok() : status(400)); + const result = await f.probe({ name: "fixture", model: "test-model", efforts: ["high"], confirmMetered: true }); + expect(result.body).toEqual({ applicable: true, sent: true, metered: true, provider: "fixture", model: "test-model", + baseline: { status: "accepted" }, image: { status: "accepted" }, reasoning: { accepted: [], rejected: ["high"], inconclusive: [] } }); + const [image, reasoning] = bodies(f.calls); + expect(image.messages).toEqual([{ role: "user", content: [{ type: "text", text: COMPLETION_TEST_PROMPT }, + { type: "image_url", image_url: { url: COMPLETION_TEST_IMAGE_DATA_URL } }] }]); + expect(image.max_tokens).toBe(32); + expect(reasoning).toMatchObject({ messages: [{ role: "user", content: COMPLETION_TEST_PROMPT }], reasoning_effort: "high" }); + expect(JSON.stringify(result.body)).not.toContain("upstream detail"); + }); + test("responses wire uses input_image and reasoning.effort", async () => { + const done = frame({ type: "response.output_text.delta", delta: "OK" }) + frame({ type: "response.completed", response: {} }); + const f = probeFixture(() => new Response(done), { modelAdapters: { "test-model": "openai-responses" } }, () => new Response(done)); + await f.probe({ name: "fixture", model: "test-model", efforts: ["low"], confirmMetered: true }); + const [image, reasoning] = bodies(f.calls); + expect(image.input).toEqual([{ role: "user", content: [{ type: "input_text", text: COMPLETION_TEST_PROMPT }, + { type: "input_image", image_url: COMPLETION_TEST_IMAGE_DATA_URL }] }]); + expect(reasoning).toMatchObject({ input: COMPLETION_TEST_PROMPT, reasoning: { effort: "low" }, max_output_tokens: 32 }); + }); + test("image 422 is rejected with code and status only", async () => { + const f = probeFixture(() => status(422)); + const result = await f.probe({ name: "fixture", model: "test-model", checks: ["image"], confirmMetered: true }); + expect(result.body.image).toEqual({ status: "rejected", error: { code: "http_status", status: 422 } }); + }); + test("image timeout is inconclusive", async () => { + const hang: Respond = (_body, init) => new Promise((_resolve, reject) => { + init!.signal!.addEventListener("abort", () => reject(init!.signal!.reason), { once: true }); + }); + const f = probeFixture(hang); + const result = await f.probe({ name: "fixture", model: "test-model", checks: ["image"], confirmMetered: true }, { timeoutMs: 30 }); + expect(result.body.image).toEqual({ status: "inconclusive", error: { code: "timeout" } }); + // A baseline that times out makes every variant inconclusive without sending it. + const g = probeFixture(ok, {}, hang); + const blocked = await g.probe({ name: "fixture", model: "test-model", checks: ["image"], confirmMetered: true }, { timeoutMs: 30 }); + expect(blocked.body).toMatchObject({ baseline: { status: "inconclusive", error: { code: "timeout" } }, image: { status: "inconclusive" } }); + expect(g.calls).toHaveLength(1); + }); + test("reasoning mix: no_content counts as accepted, 400 rejected, 5xx inconclusive", async () => { + const f = probeFixture(body => effortOf(body) === "low" ? ok() + : effortOf(body) === "medium" ? new Response(frame({ model: "test-model", choices: [{ delta: {}, finish_reason: "length" }] }) + frame("[DONE]")) + : effortOf(body) === "high" ? status(400) : status(503)); + const result = await f.probe({ name: "fixture", model: "test-model", checks: ["reasoning"], + efforts: ["xhigh", "high", "medium", "low"], confirmMetered: true }); + expect(result.body.reasoning).toEqual({ accepted: ["low", "medium"], rejected: ["high"], inconclusive: ["xhigh"] }); + expect(bodies(f.calls).map(effortOf)).toEqual(["low", "medium", "high", "xhigh"]); + }); + test("401 stops early and marks the rest inconclusive", async () => { + const f = probeFixture(body => isImage(body) ? ok() : status(401)); + const result = await f.probe({ name: "fixture", model: "test-model", efforts: ["low", "medium", "high"], confirmMetered: true }); + expect(result.body.reasoning).toEqual({ accepted: [], rejected: [], inconclusive: ["low", "medium", "high"] }); + expect(f.calls).toHaveLength(3); + }); + test("a 400 on the plain baseline is not blamed on the variant: nothing is rejected or applied", async () => { + const f = probeFixture(() => status(400), {}, () => status(400)); + const result = await f.probe({ name: "fixture", model: "test-model", efforts: ["low"], confirmMetered: true, apply: true }); + expect(result.body).toMatchObject({ baseline: { status: "rejected", error: { code: "http_status", status: 400 } }, + image: { status: "inconclusive" }, reasoning: { accepted: [], rejected: [], inconclusive: ["low"] }, applied: {} }); + expect(f.calls).toHaveLength(1); + expect(f.saves).toHaveLength(0); + }); + test("default efforts come from the model's configured ladder", async () => { + const f = probeFixture(ok, { modelReasoningEfforts: { "test-model": ["high", "low"] } }); + const result = await f.probe({ name: "fixture", model: "test-model", checks: ["reasoning"], confirmMetered: true }); + expect(result.body.reasoning.accepted).toEqual(["low", "high"]); + }); +}); + +describe("capability probe: apply", () => { + test("without apply nothing is written", async () => { + const f = probeFixture(ok); + await f.probe({ name: "fixture", model: "test-model", efforts: ["low"], confirmMetered: true }); + expect(f.saves).toHaveLength(0); + expect(f.row.modelCapabilityProbes).toBeUndefined(); + expect(f.convergences()).toBe(0); + }); + test("apply writes conclusive axes with provenance, leaves other axes, and converges", async () => { + const f = probeFixture(body => isImage(body) ? status(400) : effortOf(body) === "high" ? status(400) : ok(), { + modelCapabilities: { "test-model": { contextTier: "long_context", inputModalities: ["text", "image"] }, other: { inputModalities: ["text"] } }, + modelDefaultReasoningEfforts: { "test-model": "high" }, + }); + const result = await f.probe({ name: "fixture", model: "test-model", efforts: ["low", "high"], confirmMetered: true, apply: true }); + expect(result).toMatchObject({ status: 200, body: { applied: { inputModalities: ["text"], reasoningEfforts: ["low"] } } }); + const row = f.config.providers.fixture!; + expect(row.modelCapabilities).toEqual({ "test-model": { contextTier: "long_context", inputModalities: ["text"] }, other: { inputModalities: ["text"] } }); + expect(row.modelReasoningEfforts).toEqual({ "test-model": ["low"] }); + expect(row.modelDefaultReasoningEfforts).toBeUndefined(); + expect(row.modelCapabilityProbes).toEqual({ "test-model": { probedAt: expect.stringMatching(/Z$/), inputModalities: true, reasoningEfforts: true } }); + expect(f.saves).toHaveLength(1); + expect(f.saves[0]!.providers.fixture!.modelCapabilityProbes).toEqual(row.modelCapabilityProbes); + expect(f.convergences()).toBe(1); + }); + test("an inconclusive effort keeps the ladder untouched; a conclusive image still applies", async () => { + const f = probeFixture(body => isImage(body) ? ok() : effortOf(body) === "high" ? status(500) : ok(), + { modelReasoningEfforts: { "test-model": ["medium"] } }); + const result = await f.probe({ name: "fixture", model: "test-model", efforts: ["low", "high"], confirmMetered: true, apply: true }); + expect(result.body.applied).toEqual({ inputModalities: ["text", "image"] }); + const row = f.config.providers.fixture!; + expect(row.modelReasoningEfforts).toEqual({ "test-model": ["medium"] }); + expect(row.modelCapabilityProbes).toEqual({ "test-model": { probedAt: expect.any(String), inputModalities: true } }); + }); + test("nothing conclusive: applied is empty and nothing is saved", async () => { + const f = probeFixture(() => status(401)); + const result = await f.probe({ name: "fixture", model: "test-model", confirmMetered: true, apply: true }); + expect(result.body.baseline).toEqual({ status: "accepted" }); + expect(result.body.applied).toEqual({}); + expect(f.saves).toHaveLength(0); + expect(f.row.modelCapabilityProbes).toBeUndefined(); + }); +}); diff --git a/tests/server/price-catalog-management-api.test.ts b/tests/server/price-catalog-management-api.test.ts new file mode 100644 index 00000000000..77fdd660e67 --- /dev/null +++ b/tests/server/price-catalog-management-api.test.ts @@ -0,0 +1,211 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { join } from "node:path"; +import { clearModelCache } from "../../src/codex/model-cache"; +import { resetCodexModelEntitlementCacheForTests } from "../../src/codex/model-entitlements"; +import { saveConfig } from "../../src/config"; +import { sanitizeModelCostsForLoad } from "../../src/config/load-degrade"; +import { handleModelRoutes } from "../../src/server/management/model-routes"; +import { listManagementModelRows } from "../../src/server/management/model-rows"; +import type { OcxConfig } from "../../src/types"; +import { refreshPriceCatalogs } from "../../src/usage/price-catalog"; +import { refreshUserCostOverlays } from "../../src/usage/user-cost-overlays"; +import { installCatalogSandbox, MODELS_DEV_URL, OPENROUTER_URL } from "../helpers/price-catalog-fixtures"; + +const sandbox = installCatalogSandbox(); +const PROVIDER = "relay-test"; +let previousCodexHome: string | undefined; + +beforeEach(() => { + previousCodexHome = process.env["CODEX_HOME"]; + process.env["CODEX_HOME"] = join(sandbox.dir, "codex"); +}); +afterEach(() => { + clearModelCache(); + resetCodexModelEntitlementCacheForTests(); + if (previousCodexHome === undefined) delete process.env["CODEX_HOME"]; + else process.env["CODEX_HOME"] = previousCodexHome; +}); + +function fixture(extra: Record = {}, models = ["zephyr-flash-2"]): OcxConfig { + return { + port: 10100, + defaultProvider: PROVIDER, + modelCacheTtlMs: 60_000, + providers: { [PROVIDER]: { adapter: "openai-chat", baseUrl: "https://relay.example.invalid/v1", liveModels: false, models, ...extra } }, + } as unknown as OcxConfig; +} + +function harness(config: OcxConfig, failPersist = false) { + const persisted: OcxConfig[] = []; + async function call(method: string, path: string, body?: unknown) { + const url = new URL(`http://127.0.0.1:10100${path}`); + const response = await handleModelRoutes({ + version: "test", + req: new Request(url, { method, headers: { "Content-Type": "application/json" }, ...(body === undefined ? {} : { body: typeof body === "string" ? body : JSON.stringify(body) }) }), + url, + config, + deps: { saveConfigPreservingClaudeCode: saved => { if (failPersist) throw new Error("disk full"); persisted.push(structuredClone(saved)); } }, + convergeCodexCatalog: async () => { throw new Error("price writes must not converge catalogs"); }, + syncClaudeAgentDefsBestEffort: async () => {}, + }); + if (!response) throw new Error(`${method} ${path} was not dispatched`); + return response; + } + return { call, config, persisted }; +} + +/** Row building reloads the config from disk and rebuilds the price registries, so persist it first. */ +async function rowsFor(config: OcxConfig) { + saveConfig(config); + refreshUserCostOverlays(config); + return listManagementModelRows(config, { entitlementWaitMs: 0 }); +} + +const multiplierPath = `/api/providers/${PROVIDER}/price-multiplier`; + +describe("PUT /api/providers/:name/price-multiplier", () => { + test("round trip: set, read back, replace, and null deletes the key", async () => { + const h = harness(fixture()); + const provider = () => h.config.providers[PROVIDER]!; + expect(Object.hasOwn(provider(), "priceMultiplier")).toBe(false); + expect(await (await h.call("GET", `/api/providers/${PROVIDER}/model-costs`)).json()).toEqual({ provider: PROVIDER, modelCosts: {} }); + + const set = await h.call("PUT", multiplierPath, { multiplier: 1.5 }); + expect(set.status).toBe(200); + expect(await set.json()).toEqual({ ok: true, provider: PROVIDER, multiplier: 1.5 }); + expect(provider().priceMultiplier).toBe(1.5); + expect(h.persisted.at(-1)!.providers[PROVIDER]!.priceMultiplier).toBe(1.5); + expect(await (await h.call("GET", `/api/providers/${PROVIDER}/model-costs`)).json()).toMatchObject({ priceMultiplier: 1.5 }); + + expect((await h.call("PUT", multiplierPath, { multiplier: 0.25 })).status).toBe(200); + expect(provider().priceMultiplier).toBe(0.25); + + const cleared = await h.call("PUT", multiplierPath, { multiplier: null }); + expect(await cleared.json()).toEqual({ ok: true, provider: PROVIDER, multiplier: null }); + expect(Object.hasOwn(provider(), "priceMultiplier")).toBe(false); + expect(Object.hasOwn(h.persisted.at(-1)!.providers[PROVIDER]!, "priceMultiplier")).toBe(false); + expect(Object.hasOwn(await (await h.call("GET", `/api/providers/${PROVIDER}/model-costs`)).json(), "priceMultiplier")).toBe(false); + expect(h.persisted).toHaveLength(3); + }); + + test("clearing an already-unset multiplier succeeds and keeps the key absent", async () => { + const h = harness(fixture()); + expect((await h.call("PUT", multiplierPath, { multiplier: null })).status).toBe(200); + expect(Object.hasOwn(h.config.providers[PROVIDER]!, "priceMultiplier")).toBe(false); + }); + + test.each([ + ["zero", { multiplier: 0 }], + ["negative", { multiplier: -2 }], + ["above the cap", { multiplier: 1001 }], + ["a numeric string", { multiplier: "2" }], + ["a boolean", { multiplier: true }], + ["missing", {}], + ["an extra key", { multiplier: 2, note: "x" }], + ["an array body", [2]], + ])("%s is rejected and nothing is stored", async (_label, body) => { + const h = harness(fixture({ priceMultiplier: 3 })); + const response = await h.call("PUT", multiplierPath, body); + expect(response.status).toBe(400); + expect(h.config.providers[PROVIDER]!.priceMultiplier).toBe(3); + expect(h.persisted).toHaveLength(0); + }); + + test("the cap itself is accepted, and a raw NaN or Infinity token is invalid JSON", async () => { + const h = harness(fixture()); + expect((await h.call("PUT", multiplierPath, { multiplier: 1000 })).status).toBe(200); + expect((await h.call("PUT", multiplierPath, '{"multiplier": NaN}')).status).toBe(400); + expect((await h.call("PUT", multiplierPath, '{"multiplier": 1e999}')).status).toBe(400); + expect(h.config.providers[PROVIDER]!.priceMultiplier).toBe(1000); + }); + + test("an unknown provider is 404 and a failed persist rolls the value back", async () => { + const h = harness(fixture({ priceMultiplier: 2 }), true); + expect((await h.call("PUT", "/api/providers/nope/price-multiplier", { multiplier: 2 })).status).toBe(404); + await expect(h.call("PUT", multiplierPath, { multiplier: 5 })).rejects.toThrow("disk full"); + expect(h.config.providers[PROVIDER]!.priceMultiplier).toBe(2); + const unset = harness(fixture(), true); + await expect(unset.call("PUT", multiplierPath, { multiplier: 5 })).rejects.toThrow("disk full"); + expect(Object.hasOwn(unset.config.providers[PROVIDER]!, "priceMultiplier")).toBe(false); + }); + + test("a saved multiplier changes the price the estimator uses", async () => { + const config = fixture(); + const h = harness(config); + await refreshPriceCatalogs(); + await h.call("PUT", multiplierPath, { multiplier: 4 }); + const rows = await rowsFor(config); + expect(rows.find(row => row.provider === PROVIDER && row.id === "zephyr-flash-2")?.price) + .toMatchObject({ source: "models-dev", multiplier: 4, input: 0.4 }); + }); +}); + +describe("config load", () => { + test("an invalid priceMultiplier is dropped with a warning and a valid one is kept", () => { + const warnings: string[] = []; + const parsed = { providers: { + bad0: { priceMultiplier: 0 }, bigger: { priceMultiplier: 5000 }, text: { priceMultiplier: "2" }, nan: { priceMultiplier: Number.NaN }, + good: { priceMultiplier: 2 }, none: {}, + } }; + sanitizeModelCostsForLoad(parsed, message => warnings.push(message)); + const providers = parsed.providers as Record>; + for (const name of ["bad0", "bigger", "text", "nan"]) expect(Object.hasOwn(providers[name]!, "priceMultiplier"), name).toBe(false); + expect(providers["good"]!["priceMultiplier"]).toBe(2); + expect(warnings).toHaveLength(4); + }); +}); + +describe("POST /api/pricing/refresh", () => { + test("forces both sources and reports per-source status", async () => { + const h = harness(fixture()); + await refreshPriceCatalogs(); + const response = await h.call("POST", "/api/pricing/refresh", {}); + expect(response.status).toBe(200); + const body = await response.json(); + expect(body.ok).toBe(true); + expect(body.sources["models-dev"]).toMatchObject({ ok: true, count: 4 }); + expect(body.sources["openrouter"]).toMatchObject({ ok: true, count: 2 }); + expect(sandbox.feeds.callsTo(MODELS_DEV_URL)).toBe(2); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(2); + }); + + test("a failing source reports a short code and no response body", async () => { + const h = harness(fixture()); + sandbox.feeds.failing.add(OPENROUTER_URL); + const body = await (await h.call("POST", "/api/pricing/refresh", {})).json(); + expect(body.sources["models-dev"].ok).toBe(true); + expect(body.sources["openrouter"]).toEqual({ ok: false, error: "fetch_failed" }); + }); +}); + +describe("/api/models row price", () => { + async function priceOf(config: OcxConfig, id: string) { + const rows = await rowsFor(config); + const row = rows.find(candidate => candidate.provider === PROVIDER && candidate.id === id); + expect(row, id).toBeDefined(); + return row!.price; + } + + test("reports manual, official, models-dev and openrouter sources, and omits an unknown price", async () => { + const config = fixture({ modelCosts: { "manual-model-1": { input: 1.25, output: 5, cacheRead: 0.125, cacheWrite: 2 } } }, + ["manual-model-1", "gpt-6.1-terra", "zephyr-flash-2", "orb-only-7", "mystery-model-0"]); + await refreshPriceCatalogs(); + expect(await priceOf(config, "manual-model-1")).toEqual({ source: "manual", input: 1.25, output: 5, cacheRead: 0.125, cacheWrite: 2 }); + expect((await priceOf(config, "gpt-6.1-terra"))?.source).toBe("official"); + const dev = await priceOf(config, "zephyr-flash-2"); + expect(dev).toMatchObject({ source: "models-dev", input: 0.1, output: 0.4, cacheRead: 0.01, cacheWrite: 0.2 }); + expect(Date.parse(dev!.fetchedAt!)).not.toBeNaN(); + expect(await priceOf(config, "orb-only-7")).toMatchObject({ source: "openrouter" }); + const rows = await rowsFor(config); + const mystery = rows.find(row => row.provider === PROVIDER && row.id === "mystery-model-0")!; + expect(Object.hasOwn(mystery, "price")).toBe(false); + }); + + test("effective rates include the multiplier and a subscription provider is labelled", async () => { + const config = fixture({ priceMultiplier: 2 }); + await refreshPriceCatalogs(); + expect(await priceOf(config, "zephyr-flash-2")).toMatchObject({ source: "models-dev", input: 0.2, multiplier: 2 }); + const sub = fixture({ authMode: "oauth" }); + expect(await priceOf(sub, "zephyr-flash-2")).toMatchObject({ source: "models-dev", apiEquivalent: true }); + }); +}); diff --git a/tests/server/provider-capability-probe-provenance.test.ts b/tests/server/provider-capability-probe-provenance.test.ts new file mode 100644 index 00000000000..8e46e16060c --- /dev/null +++ b/tests/server/provider-capability-probe-provenance.test.ts @@ -0,0 +1,110 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadConfig, saveConfig } from "../../src/config"; +import { sanitizeCapabilityDeclarationsForLoad } from "../../src/config/load-degrade"; +import { modelCapabilityProbesConfigError, pruneModelCapabilityProbes } from "../../src/config/provider-validation"; +import { handleManagementAPI } from "../../src/server/management-api"; +import type { OcxConfig, OcxProviderConfig } from "../../src/types"; +import { catalogConvergenceFactory } from "../helpers/catalog-convergence"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +const AT = "2026-11-13T10:00:00.000Z"; + +describe("modelCapabilityProbes validation", () => { + test("accepts sparse rows with true axis flags and an ISO UTC timestamp", () => { + expect(modelCapabilityProbesConfigError(undefined)).toBeNull(); + expect(modelCapabilityProbesConfigError({ m: { probedAt: AT, inputModalities: true } })).toBeNull(); + expect(modelCapabilityProbesConfigError({ m: { probedAt: "2026-11-13T10:00:00Z", reasoningEfforts: true } })).toBeNull(); + }); + for (const value of [[], "x", { m: null }, { m: { probedAt: AT } }, { m: { probedAt: AT, inputModalities: false } }, + { m: { probedAt: "yesterday", inputModalities: true } }, { m: { probedAt: AT, inputModalities: true, extra: true } }, + { " m": { probedAt: AT, inputModalities: true } }, JSON.parse(`{"__proto__":{"probedAt":"${AT}","inputModalities":true}}`)]) { + test(`rejects ${JSON.stringify(value)}`, () => expect(modelCapabilityProbesConfigError(value)).not.toBeNull()); + } + test("load degradation drops malformed provenance only", () => { + const parsed = { providers: { p: { adapter: "openai-chat", modelCapabilityProbes: { m: { probedAt: "bad", inputModalities: true } } } } }; + const warnings: string[] = []; + sanitizeCapabilityDeclarationsForLoad(parsed, message => warnings.push(message)); + expect(parsed.providers.p).toEqual({ adapter: "openai-chat" } as any); + expect(warnings).toHaveLength(1); + }); + test("prune keeps only flags whose declaration exists and removes the empty field", () => { + const row = { adapter: "openai-chat", baseUrl: "https://example.test/v1", + modelCapabilities: { a: { inputModalities: ["text"] } }, modelReasoningEfforts: { a: ["low"] }, + modelCapabilityProbes: { a: { probedAt: AT, inputModalities: true, reasoningEfforts: true }, b: { probedAt: AT, inputModalities: true } }, + } as OcxProviderConfig; + const stored = row.modelCapabilityProbes; + expect(pruneModelCapabilityProbes(row, [["a", "reasoningEfforts"]])).toBe(true); + expect(row.modelCapabilityProbes).toEqual({ a: { probedAt: AT, inputModalities: true } }); + expect(stored!.a!.reasoningEfforts).toBe(true); + pruneModelCapabilityProbes(row, [["a", "inputModalities"]]); + expect("modelCapabilityProbes" in row).toBe(false); + }); +}); + +describe("modelCapabilityProbes management round trip", () => { + let home: string; + let previous: Record; + beforeEach(() => { + previous = { OPENCODEX_HOME: process.env.OPENCODEX_HOME, CODEX_HOME: process.env.CODEX_HOME }; + home = mkdtempSync(join(tmpdir(), "ocx-probe-provenance-")); + process.env.OPENCODEX_HOME = home; + process.env.CODEX_HOME = join(home, "codex"); + }); + afterEach(() => { + for (const [key, value] of Object.entries(previous)) { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + removeTreeWithRetry(home); + }); + const fixture = (): OcxConfig => ({ port: 0, defaultProvider: "caps", providers: { caps: { + adapter: "openai-chat", baseUrl: "https://example.test/v1", liveModels: false, models: ["m", "n"], + modelCapabilities: { m: { inputModalities: ["text", "image"], contextTier: "long_context" }, n: { inputModalities: ["text"] } }, + modelReasoningEfforts: { m: ["low", "high"] }, + modelCapabilityProbes: { m: { probedAt: AT, inputModalities: true, reasoningEfforts: true }, n: { probedAt: AT, inputModalities: true } }, + } } }); + const request = async (live: OcxConfig, method: string, path: string, body?: unknown) => { + const url = new URL(`http://127.0.0.1${path}`); + const response = (await handleManagementAPI(new Request(url, { method, headers: { Host: "127.0.0.1", "content-type": "application/json" }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }) }), url, live, + { createManagementConvergeCodex: catalogConvergenceFactory() }))!; + return { status: response.status, body: await response.json() as any }; + }; + + test("GET exposes it, unrelated PATCH keeps it, manual modality PATCH drops only that axis", async () => { + const live = fixture(); + saveConfig(live); + const listed = await request(live, "GET", "/api/providers"); + expect(listed.body.find((row: any) => row.name === "caps").modelCapabilityProbes).toEqual(fixture().providers.caps!.modelCapabilityProbes); + expect((await request(live, "PATCH", "/api/providers?name=caps", { contextWindow: 64000 })).status).toBe(200); + expect(loadConfig().providers.caps!.modelCapabilityProbes).toEqual(fixture().providers.caps!.modelCapabilityProbes); + expect((await request(live, "PATCH", "/api/providers?name=caps", { modelCapabilities: { m: { inputModalities: ["text"] } } })).status).toBe(200); + expect(loadConfig().providers.caps!.modelCapabilityProbes).toEqual({ + m: { probedAt: AT, reasoningEfforts: true }, n: { probedAt: AT, inputModalities: true } }); + expect((await request(live, "PATCH", "/api/providers?name=caps", { modelCapabilities: null })).status).toBe(200); + expect(loadConfig().providers.caps!.modelCapabilityProbes).toEqual({ m: { probedAt: AT, reasoningEfforts: true } }); + }); + + test("same-name POST carries it when omitted and rejects a malformed value", async () => { + const live = fixture(); + saveConfig(live); + const form = { adapter: "openai-chat", baseUrl: "https://example.test/v1", liveModels: false, models: ["m", "n"] }; + expect((await request(live, "POST", "/api/providers", { name: "caps", provider: { ...form, modelCapabilityProbes: { m: { probedAt: "x" } } } })).status).toBe(400); + expect((await request(live, "POST", "/api/providers", { name: "caps", provider: form })).status).toBe(200); + // modelCapabilities is carried; modelReasoningEfforts is not, so its flag goes with it. + expect(loadConfig().providers.caps!.modelCapabilityProbes).toEqual({ m: { probedAt: AT, inputModalities: true }, n: { probedAt: AT, inputModalities: true } }); + }); + + test("model-settings edit of a probed ladder makes it manual and stays sparse", async () => { + const live = fixture(); + live.providers.caps!.modelCapabilityProbes = { m: { probedAt: AT, reasoningEfforts: true } }; + saveConfig(live); + const response = await request(live, "PUT", "/api/model-settings", { provider: "caps", modelId: "m", reasoningEfforts: ["low", "high"] }); + expect(response.status).toBe(200); + expect(response.body).toMatchObject({ changed: true, saved: true }); + expect("modelCapabilityProbes" in loadConfig().providers.caps!).toBe(false); + }); +}); diff --git a/tests/usage/price-catalog-miss.test.ts b/tests/usage/price-catalog-miss.test.ts new file mode 100644 index 00000000000..d27f9fc8af1 --- /dev/null +++ b/tests/usage/price-catalog-miss.test.ts @@ -0,0 +1,155 @@ +import { describe, expect, setSystemTime, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { settleBackgroundWork } from "../../src/lib/background-work"; +import { resolveMatchedPrice } from "../../src/usage/cost"; +import { + lookupCatalogPrice, + notePriceCatalogMiss, + PRICE_CATALOG_MISS_COOLDOWN_MS, + refreshPriceCatalogs, + resetPriceCatalogForTests, + resolveCatalogPrice, + setPriceCatalogFetchForTests, +} from "../../src/usage/price-catalog"; +import { feedFetch, installCatalogSandbox, MODELS_DEV_URL, nextTick } from "../helpers/price-catalog-fixtures"; + +const sandbox = installCatalogSandbox(); + +/** A miss flushes on a zero-delay timer; let it start, then wait for the fetch it began. */ +async function drainMisses(): Promise { + await nextTick(); + await settleBackgroundWork(); +} + +describe("price catalog miss path", () => { + test("a miss triggers one background fetch per source and the next lookup hits", async () => { + expect(resolveCatalogPrice("zephyr-flash-2")).toBeNull(); + expect(sandbox.feeds.calls).toHaveLength(0); + await drainMisses(); + expect(sandbox.feeds.calls.sort()).toEqual(["https://models.dev/api.json", "https://openrouter.ai/api/v1/models"]); + expect(resolveCatalogPrice("zephyr-flash-2")?.source).toBe("models-dev"); + }); + + test("many misses in one tick share one fetch per source", async () => { + for (const id of ["zephyr-flash-2", "moon-pro-3.1", "orb-only-7", "cn:grok-quill-5.5", "nothing-1"]) resolveCatalogPrice(id); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(2); + expect(lookupCatalogPrice("orb-only-7")?.source).toBe("openrouter"); + expect(lookupCatalogPrice("moon-pro-3-1")?.source).toBe("models-dev"); + }); + + test("the same key queued twice in one tick is fetched once", async () => { + notePriceCatalogMiss("zephyr-flash-2"); + notePriceCatalogMiss("ZEPHYR.FLASH.2"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(2); + }); + + test("resolveMatchedPrice returns null at once while the fetch is still pending, then prices it", async () => { + const release = sandbox.feeds.hold(); + const first = resolveMatchedPrice("acme-relay", "zephyr-flash-2"); + expect(first).toBeNull(); + await nextTick(); + expect(sandbox.feeds.calls).toHaveLength(1); + expect(resolveMatchedPrice("acme-relay", "zephyr-flash-2")).toBeNull(); + release(); + await settleBackgroundWork(); + expect(sandbox.feeds.calls).toHaveLength(2); + expect(resolveMatchedPrice("acme-relay", "zephyr-flash-2")).toMatchObject({ + source: "models-dev", + status: "verified-derived", + cost4: { input: 0.1, output: 0.4 }, + }); + }); + + test("a key still missing afterwards cools down: no second fetch for it", async () => { + resolveCatalogPrice("nothing-9"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(2); + resolveCatalogPrice("nothing-9"); + resolveCatalogPrice("NOTHING.9"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(2); + expect(JSON.parse(readFileSync(join(sandbox.dir, "price-catalog-cache.json"), "utf8")).missCooldown).toHaveProperty("nothing-9"); + }); + + test("the cooldown survives a restart and expires after six hours", async () => { + const start = Date.now(); + resolveCatalogPrice("nothing-9"); + await drainMisses(); + resetPriceCatalogForTests(); + const feeds = feedFetch(); + setPriceCatalogFetchForTests(feeds.fetch); + resolveCatalogPrice("nothing-9"); + await drainMisses(); + expect(feeds.calls).toHaveLength(0); + setSystemTime(start + PRICE_CATALOG_MISS_COOLDOWN_MS + 60_000); + resolveCatalogPrice("nothing-9"); + await drainMisses(); + expect(feeds.calls).toHaveLength(2); + }); + + test("a fresh source gets at most one miss-driven fetch per cooldown window", async () => { + const start = Date.now(); + await refreshPriceCatalogs(); + sandbox.feeds.calls.length = 0; + resolveCatalogPrice("late-model-4"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(2); + resolveCatalogPrice("late-model-5"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(2); + setSystemTime(start + PRICE_CATALOG_MISS_COOLDOWN_MS + 60_000); + resolveCatalogPrice("late-model-5"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(4); + }); + + test("a miss-driven fetch merges only the missing keys and leaves other rows untouched", async () => { + await refreshPriceCatalogs(); + const fetchedAt = JSON.parse(readFileSync(join(sandbox.dir, "price-catalog-cache.json"), "utf8")).sources["models-dev"].fetchedAt; + sandbox.feeds.serve(MODELS_DEV_URL, { deepseek: { models: { + "zephyr-flash-2": { cost: { input: 77, output: 88 } }, + "late-model-4": { cost: { input: 3, output: 4 } }, + "unrelated-5": { cost: { input: 5, output: 6 } }, + } } }); + resolveCatalogPrice("late-model-4"); + await drainMisses(); + expect(lookupCatalogPrice("late-model-4")?.cost4).toMatchObject({ input: 3, output: 4 }); + expect(lookupCatalogPrice("zephyr-flash-2")?.cost4).toMatchObject({ input: 0.1, output: 0.4 }); + expect(lookupCatalogPrice("moon-pro-3-1")?.cost4.input).toBe(1.5); + expect(lookupCatalogPrice("unrelated-5")).toBeNull(); + const file = JSON.parse(readFileSync(join(sandbox.dir, "price-catalog-cache.json"), "utf8")); + expect(file.sources["models-dev"].fetchedAt).toBe(fetchedAt); + }); + + test("a forced refresh clears cooldowns so a still-missing key is asked for again", async () => { + resolveCatalogPrice("nothing-9"); + await drainMisses(); + sandbox.feeds.calls.length = 0; + await refreshPriceCatalogs({ force: true }); + expect(sandbox.feeds.calls).toHaveLength(2); + resolveCatalogPrice("nothing-9"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(4); + }); + + test("a failed miss fetch records no cooldown, so the key is retried later", async () => { + sandbox.feeds.failing.add(MODELS_DEV_URL); + sandbox.feeds.failing.add("https://openrouter.ai/api/v1/models"); + resolveCatalogPrice("zephyr-flash-2"); + await drainMisses(); + expect(lookupCatalogPrice("zephyr-flash-2")).toBeNull(); + sandbox.feeds.failing.clear(); + await refreshPriceCatalogs({ force: true }); + expect(lookupCatalogPrice("zephyr-flash-2")?.source).toBe("models-dev"); + }); + + test("without an injected fetch the test environment never reaches the network", async () => { + setPriceCatalogFetchForTests(null); + resolveCatalogPrice("zephyr-flash-2"); + await drainMisses(); + expect(sandbox.feeds.calls).toHaveLength(0); + }); +}); diff --git a/tests/usage/price-catalog-resolution.test.ts b/tests/usage/price-catalog-resolution.test.ts new file mode 100644 index 00000000000..52738c91ed0 --- /dev/null +++ b/tests/usage/price-catalog-resolution.test.ts @@ -0,0 +1,184 @@ +import { describe, expect, test } from "bun:test"; +import { costResult } from "../../src/server/management/shared"; +import { estimateRequestCost, resolveMatchedPrice } from "../../src/usage/cost"; +import { refreshPriceCatalogs } from "../../src/usage/price-catalog"; +import { refreshUserCostOverlays } from "../../src/usage/user-cost-overlays"; +import type { OcxConfig } from "../../src/types"; +import { installCatalogSandbox, MODELS_DEV_URL } from "../helpers/price-catalog-fixtures"; + +const sandbox = installCatalogSandbox(); +const USAGE = { inputTokens: 1_000_000, outputTokens: 1_000_000, cachedInputTokens: 0, cacheCreationInputTokens: 0 }; +const ZERO = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }; + +function configure(providers: Record>): void { + refreshUserCostOverlays({ + port: 0, + defaultProvider: Object.keys(providers)[0] ?? "none", + providers: Object.fromEntries(Object.entries(providers).map(([name, extra]) => + [name, { adapter: "openai-chat", baseUrl: "https://relay.example.invalid/v1", models: [], ...extra }])), + } as unknown as OcxConfig); +} + +async function loadCatalog(): Promise { + sandbox.feeds.serve(MODELS_DEV_URL, { + deepseek: { models: { "zephyr-flash-2": { cost: { input: 0.1, output: 0.4 } } } }, + openai: { models: { "gpt-6.1-terra": { cost: { input: 99, output: 99 } } } }, + }); + await refreshPriceCatalogs(); +} + +describe("price resolution tiers", () => { + test("a catalog price is a verified-derived estimate and names its source", async () => { + configure({ acme: {} }); + await loadCatalog(); + const price = resolveMatchedPrice("acme", "zephyr-flash-2")!; + expect(price).toMatchObject({ source: "models-dev", status: "verified-derived", cost4: { input: 0.1, output: 0.4 }, sourceRef: "https://models.dev/api.json" }); + expect(Date.parse(price.verifiedAt!)).not.toBeNaN(); + const estimate = estimateRequestCost({ provider: "acme", model: "zephyr-flash-2", usage: USAGE, usageStatus: "reported" })!; + expect(estimate.estimated).toBe(true); + expect(estimate.cost.total).toBeCloseTo(0.5, 9); + expect(resolveMatchedPrice("acme", "unknown-model-9")).toBeNull(); + }); + + test("OpenRouter prices a model models.dev does not list", async () => { + configure({ acme: {} }); + await refreshPriceCatalogs(); + expect(resolveMatchedPrice("acme", "orb-only-7")).toMatchObject({ source: "openrouter", status: "verified-derived" }); + }); + + test("a manual price beats the catalog, and manual zero rates still win", async () => { + configure({ acme: { modelCosts: { "zephyr-flash-2": { input: 7, output: 9, cacheRead: 0, cacheWrite: 0 } } } }); + await loadCatalog(); + expect(resolveMatchedPrice("acme", "zephyr-flash-2")).toMatchObject({ source: "user", cost4: { input: 7, output: 9 } }); + configure({ acme: { modelCosts: { "zephyr-flash-2": ZERO } } }); + const free = resolveMatchedPrice("acme", "zephyr-flash-2")!; + expect(free.source).toBe("user"); + expect(free.cost4).toEqual(ZERO); + }); + + test("an official price beats the catalog", async () => { + configure({}); + await loadCatalog(); + const price = resolveMatchedPrice("openai-apikey", "gpt-6.1-terra")!; + expect(["expected", "jawcode"]).toContain(price.source); + expect(price.cost4.input).toBe(1); + }); + + test("allowModelLevelFallback: false skips the catalog", async () => { + configure({ acme: {} }); + await loadCatalog(); + expect(resolveMatchedPrice("acme", "zephyr-flash-2", undefined, undefined, { allowModelLevelFallback: false })).toBeNull(); + expect(resolveMatchedPrice("acme", "zephyr-flash-2")?.source).toBe("models-dev"); + }); + + test("a vendor prefix and a channel prefix on the logged id still reach the catalog price", async () => { + configure({ acme: {} }); + await loadCatalog(); + for (const id of ["cn:zephyr-flash-2", "zd/zephyr-flash-2", "Zephyr.Flash.2"]) { + expect(resolveMatchedPrice("acme", id)?.cost4.input, id).toBe(0.1); + } + }); +}); + +describe("relay price multiplier", () => { + test("scales catalog and official prices but never a manual price", async () => { + configure({ relay: { priceMultiplier: 2, modelCosts: { "manual-model-1": { input: 3, output: 4, cacheRead: 0, cacheWrite: 0 } } } }); + await loadCatalog(); + expect(resolveMatchedPrice("relay", "zephyr-flash-2")).toMatchObject({ source: "models-dev", multiplier: 2, cost4: { input: 0.2, output: 0.8 } }); + expect(resolveMatchedPrice("relay", "manual-model-1")).toMatchObject({ source: "user", cost4: { input: 3, output: 4 } }); + expect(resolveMatchedPrice("relay", "manual-model-1")!.multiplier).toBeUndefined(); + configure({ relay: { priceMultiplier: 0.5 }, plain: {} }); + expect(resolveMatchedPrice("relay", "gpt-6.1-terra")!.multiplier).toBe(0.5); + }); + + test("changing or removing the multiplier takes effect on the memoized price", async () => { + configure({ relay: { priceMultiplier: 2 } }); + await loadCatalog(); + expect(resolveMatchedPrice("relay", "zephyr-flash-2")!.cost4.input).toBeCloseTo(0.2, 12); + configure({ relay: { priceMultiplier: 10 } }); + expect(resolveMatchedPrice("relay", "zephyr-flash-2")!.cost4.input).toBeCloseTo(1, 12); + configure({ relay: {} }); + const plain = resolveMatchedPrice("relay", "zephyr-flash-2")!; + expect(plain.cost4.input).toBeCloseTo(0.1, 12); + expect(plain.multiplier).toBeUndefined(); + }); + + test("an unset multiplier, or one on another provider, changes nothing", async () => { + configure({ relay: { priceMultiplier: 4 }, plain: {} }); + await loadCatalog(); + expect(resolveMatchedPrice("plain", "zephyr-flash-2")!.cost4.input).toBe(0.1); + expect(resolveMatchedPrice("plain", "zephyr-flash-2")!.multiplier).toBeUndefined(); + }); + + test("the cost estimate uses the scaled rates", async () => { + configure({ relay: { priceMultiplier: 3 } }); + await loadCatalog(); + const estimate = estimateRequestCost({ provider: "relay", model: "zephyr-flash-2", usage: USAGE, usageStatus: "reported" })!; + expect(estimate.cost.total).toBeCloseTo(1.5, 9); + }); +}); + +describe("subscription API-equivalent label", () => { + test("oauth and forward providers and Codex account identities carry apiEquivalent; key providers do not", async () => { + configure({ + "sub-oauth": { authMode: "oauth" }, + "sub-forward": { authMode: "forward" }, + "pay-key": { authMode: "key" }, + "pay-default": {}, + }); + await loadCatalog(); + expect(resolveMatchedPrice("sub-oauth", "zephyr-flash-2")!.apiEquivalent).toBe(true); + expect(resolveMatchedPrice("sub-forward", "zephyr-flash-2")!.apiEquivalent).toBe(true); + expect(resolveMatchedPrice("openai-main", "gpt-6.1-terra")!.apiEquivalent).toBe(true); + expect(resolveMatchedPrice("pay-key", "zephyr-flash-2")!.apiEquivalent).toBeUndefined(); + expect(resolveMatchedPrice("pay-default", "zephyr-flash-2")!.apiEquivalent).toBeUndefined(); + }); + + test("a manual price on a subscription provider is not relabelled", async () => { + configure({ "sub-oauth": { authMode: "oauth", modelCosts: { "zephyr-flash-2": { input: 1, output: 2, cacheRead: 0, cacheWrite: 0 } } } }); + await loadCatalog(); + expect(resolveMatchedPrice("sub-oauth", "zephyr-flash-2")!.apiEquivalent).toBeUndefined(); + }); + + test("switching a provider's auth mode refreshes the label", async () => { + configure({ flip: { authMode: "key" } }); + await loadCatalog(); + expect(resolveMatchedPrice("flip", "zephyr-flash-2")!.apiEquivalent).toBeUndefined(); + configure({ flip: { authMode: "oauth" } }); + expect(resolveMatchedPrice("flip", "zephyr-flash-2")!.apiEquivalent).toBe(true); + }); +}); + +describe("cost result reasons", () => { + const entry = (provider: string, model: string) => + ({ provider, model, usage: USAGE, usageStatus: "reported" }) as unknown as Parameters[0]; + const reasonsOf = (provider: string, model: string): string[] => { + const result = costResult(entry(provider, model)); + if (result.kind !== "value") throw new Error(`no estimate for ${provider}/${model}`); + return result.estimateReasons; + }; + + test("a plain catalog price reports only online_price_catalog", async () => { + configure({ acme: {} }); + await loadCatalog(); + const reasons = reasonsOf("acme", "zephyr-flash-2"); + expect(reasons).toContain("online_price_catalog"); + expect(reasons).not.toContain("relay_price_multiplier"); + expect(reasons).not.toContain("subscription_api_equivalent"); + }); + + test("multiplier and subscription add their reasons", async () => { + configure({ relay: { priceMultiplier: 2 }, "sub-oauth": { authMode: "oauth" } }); + await loadCatalog(); + expect(reasonsOf("relay", "zephyr-flash-2")).toEqual(expect.arrayContaining(["online_price_catalog", "relay_price_multiplier"])); + expect(reasonsOf("sub-oauth", "zephyr-flash-2")).toEqual(expect.arrayContaining(["online_price_catalog", "subscription_api_equivalent"])); + }); + + test("a manual price carries provider_cost_overlay and none of the automatic reasons", async () => { + configure({ relay: { priceMultiplier: 2, modelCosts: { "zephyr-flash-2": { input: 1, output: 2, cacheRead: 0, cacheWrite: 0 } } } }); + await loadCatalog(); + const reasons = reasonsOf("relay", "zephyr-flash-2"); + expect(reasons).toContain("provider_cost_overlay"); + for (const reason of ["online_price_catalog", "relay_price_multiplier", "subscription_api_equivalent"]) expect(reasons).not.toContain(reason); + }); +}); diff --git a/tests/usage/price-catalog.test.ts b/tests/usage/price-catalog.test.ts new file mode 100644 index 00000000000..d1cf17a8eea --- /dev/null +++ b/tests/usage/price-catalog.test.ts @@ -0,0 +1,202 @@ +import { describe, expect, setSystemTime, test } from "bun:test"; +import { existsSync, readFileSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { + lookupCatalogPrice, + PRICE_CATALOG_TTL_MS, + priceCatalogKey, + refreshPriceCatalogs, + resetPriceCatalogForTests, + setPriceCatalogFetchForTests, +} from "../../src/usage/price-catalog"; +import { priceCatalogVersion } from "../../src/usage/price-catalog-version"; +import { feedFetch, installCatalogSandbox, MODELS_DEV_URL, OPENROUTER_URL } from "../helpers/price-catalog-fixtures"; + +const sandbox = installCatalogSandbox(); +const cacheFile = () => join(sandbox.dir, "price-catalog-cache.json"); + +describe("price catalog key matching", () => { + test.each([ + ["cn:zephyr-flash-2", "zephyr-flash-2"], + ["zd/zephyr-flash-2", "zephyr-flash-2"], + ["deepseek-ai/Zephyr-Flash-2", "zephyr-flash-2"], + ["ZEPHYR.FLASH.2", "zephyr-flash-2"], + ["cn:acme/Moon-Pro-3.1[1m]", "moon-pro-3-1"], + ])("%s collapses to %s", (id, key) => { + expect(priceCatalogKey(id)).toBe(key); + }); + + test("a channel prefix, vendor prefixes, case and dots all find the same row", async () => { + await refreshPriceCatalogs(); + for (const id of ["zephyr-flash-2", "cn:zephyr-flash-2", "zd/zephyr-flash-2", "deepseek-ai/Zephyr-Flash-2", "ZEPHYR-FLASH-2"]) { + expect(lookupCatalogPrice(id)?.vendorModelId, id).toBe("zephyr-flash-2"); + } + expect(lookupCatalogPrice("Moon-Pro-3.1")?.cost4).toEqual({ input: 1.5, output: 6, cacheRead: 0, cacheWrite: 0 }); + expect(lookupCatalogPrice("moon-pro-3-1")?.vendor).toBe("moonshotai"); + expect(lookupCatalogPrice("grok-quill-5.5")?.cost4).toEqual({ input: 2, output: 8, cacheRead: 0.5, cacheWrite: 0 }); + expect(lookupCatalogPrice("unknown-model-9")).toBeNull(); + }); + + test("a key claimed by two rows with different prices in one source answers nothing", async () => { + await refreshPriceCatalogs(); + expect(lookupCatalogPrice("twin-1")).toBeNull(); + expect(lookupCatalogPrice("dup-2")).toBeNull(); + expect(lookupCatalogPrice("same-1")?.cost4).toMatchObject({ input: 5, output: 6 }); + }); + + test("models.dev providers that are not the model's own vendor are ignored", async () => { + await refreshPriceCatalogs(); + expect(lookupCatalogPrice("reseller-only-1")).toBeNull(); + expect(lookupCatalogPrice("zephyr-flash-2")).toMatchObject({ source: "models-dev", vendor: "deepseek", cost4: { input: 0.1 } }); + }); + + test("zero, missing and negative rates are skipped", async () => { + await refreshPriceCatalogs(); + for (const id of ["gem-free-4", "gem-nocost-4", "gem-badrate-4", "negative-1"]) expect(lookupCatalogPrice(id), id).toBeNull(); + }); + + test("models.dev rates are already per 1M tokens and a missing cache rate is zero", async () => { + await refreshPriceCatalogs(); + expect(lookupCatalogPrice("zephyr-flash-2")).toMatchObject({ + source: "models-dev", + url: MODELS_DEV_URL, + cost4: { input: 0.1, output: 0.4, cacheRead: 0.01, cacheWrite: 0.2 }, + }); + }); + + test("OpenRouter per-token prices become per-1M rates", async () => { + await refreshPriceCatalogs(); + const price = lookupCatalogPrice("orb-only-7")!; + expect(price).toMatchObject({ source: "openrouter", vendor: "acme", vendorModelId: "acme/orb-only-7", url: OPENROUTER_URL }); + expect(price.cost4.input).toBeCloseTo(0.5, 9); + expect(price.cost4.output).toBeCloseTo(2, 9); + expect(price.cost4.cacheRead).toBeCloseTo(0.1, 9); + expect(price.cost4.cacheWrite).toBeCloseTo(1.25, 9); + }); + + test("OpenRouter :free variants and openrouter/ ids are ignored", async () => { + await refreshPriceCatalogs(); + expect(lookupCatalogPrice("free-only-1")).toBeNull(); + expect(lookupCatalogPrice("auto")).toBeNull(); + }); + + test("models.dev outranks OpenRouter for a model both list", async () => { + await refreshPriceCatalogs(); + expect(lookupCatalogPrice("zephyr-flash-2")?.source).toBe("models-dev"); + sandbox.feeds.serve(MODELS_DEV_URL, { deepseek: { models: { "other-3": { cost: { input: 1, output: 1 } } } } }); + await refreshPriceCatalogs({ force: true }); + expect(lookupCatalogPrice("zephyr-flash-2")?.source).toBe("openrouter"); + }); +}); + +describe("price catalog cache", () => { + test("a refresh persists both snapshots and reports per-source status", async () => { + const before = priceCatalogVersion(); + const status = await refreshPriceCatalogs(); + expect(status["models-dev"]).toMatchObject({ ok: true, count: 4 }); + expect(status["openrouter"]).toMatchObject({ ok: true, count: 2 }); + expect(Date.parse(status["models-dev"].fetchedAt!)).not.toBeNaN(); + expect(priceCatalogVersion()).toBeGreaterThan(before); + const file = JSON.parse(readFileSync(cacheFile(), "utf8")); + expect(file.version).toBe(1); + expect(Object.keys(file.sources).sort()).toEqual(["models-dev", "openrouter"]); + }); + + test("a fresh cache is not refetched, an expired one is", async () => { + const start = Date.now(); + await refreshPriceCatalogs(); + expect(sandbox.feeds.calls).toHaveLength(2); + setSystemTime(start + PRICE_CATALOG_TTL_MS - 1000); + await refreshPriceCatalogs(); + expect(sandbox.feeds.calls).toHaveLength(2); + setSystemTime(start + PRICE_CATALOG_TTL_MS + 60_000); + await refreshPriceCatalogs(); + expect(sandbox.feeds.callsTo(MODELS_DEV_URL)).toBe(2); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(2); + }); + + test("a fresh cache file from a previous process is read without any fetch", async () => { + await refreshPriceCatalogs(); + resetPriceCatalogForTests(); + const feeds = feedFetch(); + setPriceCatalogFetchForTests(feeds.fetch); + expect(lookupCatalogPrice("zephyr-flash-2")?.source).toBe("models-dev"); + await refreshPriceCatalogs(); + expect(feeds.calls).toHaveLength(0); + }); + + test("concurrent refreshes coalesce into one fetch per source", async () => { + const release = sandbox.feeds.hold(); + const runs = [refreshPriceCatalogs(), refreshPriceCatalogs(), refreshPriceCatalogs()]; + release(); + const results = await Promise.all(runs); + expect(sandbox.feeds.callsTo(MODELS_DEV_URL)).toBe(1); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(1); + for (const result of results) expect(result["models-dev"].ok).toBe(true); + }); + + test("offline keeps the previous snapshot and answers with a short error code", async () => { + const start = Date.now(); + await refreshPriceCatalogs(); + sandbox.feeds.failing.add(MODELS_DEV_URL); + sandbox.feeds.failing.add(OPENROUTER_URL); + setSystemTime(start + PRICE_CATALOG_TTL_MS + 60_000); + const status = await refreshPriceCatalogs(); + expect(status["models-dev"]).toEqual({ ok: false, error: "fetch_failed" }); + expect(lookupCatalogPrice("zephyr-flash-2")?.source).toBe("models-dev"); + expect(lookupCatalogPrice("orb-only-7")?.source).toBe("openrouter"); + }); + + test("offline with no cache stays at no catalog", async () => { + sandbox.feeds.failing.add(MODELS_DEV_URL); + sandbox.feeds.failing.add(OPENROUTER_URL); + const status = await refreshPriceCatalogs(); + expect(status["openrouter"].ok).toBe(false); + expect(lookupCatalogPrice("zephyr-flash-2")).toBeNull(); + expect(existsSync(cacheFile())).toBe(false); + }); + + test("bad JSON and an empty feed are failures with distinct short codes", async () => { + sandbox.feeds.serve(MODELS_DEV_URL, "not json {"); + sandbox.feeds.serve(OPENROUTER_URL, { data: [] }); + const status = await refreshPriceCatalogs(); + expect(status["models-dev"]).toEqual({ ok: false, error: "bad_json" }); + expect(status["openrouter"]).toEqual({ ok: false, error: "empty" }); + }); + + test("a failed source backs off for a minute but a forced refresh does not", async () => { + sandbox.feeds.failing.add(OPENROUTER_URL); + await refreshPriceCatalogs(); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(1); + expect((await refreshPriceCatalogs())["openrouter"]).toEqual({ ok: false, error: "backoff" }); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(1); + sandbox.feeds.failing.delete(OPENROUTER_URL); + const forced = await refreshPriceCatalogs({ force: true }); + expect(forced["openrouter"].ok).toBe(true); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(2); + }); + + test("a forced refresh refetches a fresh cache", async () => { + await refreshPriceCatalogs(); + await refreshPriceCatalogs({ force: true }); + expect(sandbox.feeds.callsTo(MODELS_DEV_URL)).toBe(2); + expect(sandbox.feeds.callsTo(OPENROUTER_URL)).toBe(2); + }); + + test("a corrupt cache file means no catalog and does not throw", async () => { + writeFileSync(cacheFile(), "{ this is not json"); + expect(lookupCatalogPrice("zephyr-flash-2")).toBeNull(); + resetPriceCatalogForTests(); + writeFileSync(cacheFile(), JSON.stringify({ version: 2, sources: {} })); + expect(lookupCatalogPrice("zephyr-flash-2")).toBeNull(); + }); + + test("invalid rows inside an otherwise good cache file are dropped", async () => { + writeFileSync(cacheFile(), JSON.stringify({ version: 1, sources: { "models-dev": { fetchedAt: Date.now(), prices: { + good: { cost4: { input: 1, output: 2, cacheRead: 0, cacheWrite: 0 }, vendor: "deepseek", vendorModelId: "good" }, + bad: { cost4: { input: -1, output: 2, cacheRead: 0, cacheWrite: 0 }, vendor: "deepseek", vendorModelId: "bad" }, + } } } })); + expect(lookupCatalogPrice("good")?.cost4.input).toBe(1); + expect(lookupCatalogPrice("bad")).toBeNull(); + }); +});