diff --git a/docs-site/src/content/docs/es/guides/claude-code.md b/docs-site/src/content/docs/es/guides/claude-code.md index 9ceb03b9dae..c5e49978d1d 100644 --- a/docs-site/src/content/docs/es/guides/claude-code.md +++ b/docs-site/src/content/docs/es/guides/claude-code.md @@ -822,11 +822,12 @@ Claude Code declara la herramienta tipada `advisor_*`, como `advisor_20260301` c - Otros destinos, incluidos GPT, Gemini y gateways sin esa opción, omiten la declaración y ejecutan sin asesor. No bloquea destinos de combos o alternativas. - Si el [Advisor de OpenCodex](/es/reference/configuration/advisor/) está activo, controla el nombre `advisor` y no se reenvía la herramienta declarada; el registro indica `advisor-tool-shadowed-by-proxy-advisor`. - Si la clave del cliente no admite el `model` declarado, se omite con `advisor-tool-outside-model-scope`; el carril Messages nativo administrado rechaza la solicitud. +- Si la política capturada para la solicitud tiene un límite de gasto en tokens aplicable por raíz, identidad o grupo de proveedor, se omiten la declaración del Advisor alojado, su historial completado y la beta. El turno principal continúa. En solicitudes enrutadas directamente, el registro indica `advisor-tool-withheld-by-spend-policy`. Se usa la política capturada para esa solicitud, con la cuenta y el grupo seleccionados; los cambios posteriores de configuración no alteran la decisión. Sin un límite aplicable se mantienen las reglas de reenvío anteriores. - `server_tool_use` y `advisor_tool_result` llegan sin cambios, incluidos ID `srvtoolu_`, en streaming y sin él. Incluye `advisor_result`, `advisor_redacted_result` cifrado y `advisor_tool_result_error`. El contenido cifrado no se lee, reescribe ni registra. - Al reenviar un intercambio completo en el historial, se conserva en su posición, con beta, solo hacia destinos que admiten la declaración. La entrada debe ser un objeto vacío y el resultado una de esas tres variantes; el resto se descarta. - Una llamada pendiente al terminar con `pause_turn` o `tool_use` de cliente no se muestra ni reanuda. La siguiente solicitud conserva declaración y beta para consultar de nuevo. - Si OpenCodex controla Advisor, también excluye los intercambios anteriores de Claude durante todo el turno para no cambiar el prompt a mitad. -- Los tokens del asesor se informan por separado y no se suman a los registros de uso, gasto de clave ni uso devuelto a Claude Code. Consulta su coste en Anthropic. +- OpenCodex no contabiliza por separado los tokens del asesor alojado en los registros de uso, el gasto de clave ni el uso devuelto a Claude Code. Por eso los límites de tokens omiten esta herramienta opcional. Si se reenvía sin límite, consulta su coste en Anthropic. **Afinidad OpenAI/ChatGPT nativa.** `prompt_cache_key` sirve a la afinidad de caché; `session_id` solo procede de identidad de sesión, nunca de la clave compartida. Si `metadata.user_id` es una cadena JSON con otras claves, se elimina el token por turno `tk` antes del hash, manteniendo identidad entre turnos remotos. Cadenas que no sean objetos JSON, arrays o un objeto con solo `tk` se hashean sin cambios; `user` sigue derivándose de la cadena original. Sin cadena y con sistema, se usa el hash de modelo resuelto, sistema y esquemas de herramientas: identifica un grupo de solicitudes, no una sesión. diff --git a/docs-site/src/content/docs/es/reference/configuration/providers.md b/docs-site/src/content/docs/es/reference/configuration/providers.md index 62c7a3fe9d2..8cf5b43824f 100644 --- a/docs-site/src/content/docs/es/reference/configuration/providers.md +++ b/docs-site/src/content/docs/es/reference/configuration/providers.md @@ -314,7 +314,7 @@ Los proveedores pueden exponer una abreviatura incorporada, como `agy` para `goo | `noVisionModels?` | `string[]` | Los modelos solo de texto enviados a través del sidecar de visión toleran una etiqueta Ollama `:size`. | | `escapeBuiltinToolNames?` | `boolean` | Escápese de los nombres de herramientas incorporados para las puertas de enlace compatibles con Anthropic y restaure las llamadas devueltas. | | `anthropicEofTolerance?` | `boolean` | Deje que una puerta de enlace compatible con Anthropic complete un flujo que termina antes de `message_stop`, solo cuando se recibió texto visible o una entrada completa de la herramienta JSON-objeto. Desactivado por defecto. | -| `forwardAdvisorTool?` | `boolean` | Herramienta Advisor tipada de Claude Code, `advisor_20260301`, en Messages traducido. Ausente: solo un adaptador `anthropic` con `https://api.anthropic.com` recibe declaración, intercambios completados y beta `advisor-tool-2026-03-01`. `true` los permite en un gateway Anthropic compatible que ejecute Advisor; `false` impide reenviarlos incluso al oficial. Otros destinos omiten la herramienta. Un Advisor de OpenCodex activo controla el nombre `advisor`. Consulta [Advisor de Anthropic](/es/guides/claude-code/#anthropics-advisor-tool-on-translated-routes). Guardar con el mismo destino conserva un valor omitido; moverlo lo elimina. `PATCH` acepta booleano o `null`. | +| `forwardAdvisorTool?` | `boolean` | Herramienta Advisor tipada de Claude Code, `advisor_20260301`, en Messages traducido. Ausente: solo un adaptador `anthropic` con `https://api.anthropic.com` recibe declaración, intercambios completados y beta `advisor-tool-2026-03-01`. `true` los permite en un gateway Anthropic compatible que ejecute Advisor; `false` impide reenviarlos incluso al oficial. Otros destinos omiten la herramienta. Un Advisor de OpenCodex activo controla el nombre `advisor`. Un límite de gasto en tokens aplicable a la solicitud también omite la declaración, el historial completado de Advisor y la beta. Consulta [Advisor de Anthropic](/es/guides/claude-code/#anthropics-advisor-tool-on-translated-routes). Guardar con el mismo destino conserva un valor omitido; moverlo lo elimina. `PATCH` acepta booleano o `null`. | | `googleMode?` | `"ai-studio" \| "vertex" \| "cloud-code-assist"` | Google transporte/modo auth. Por defecto `ai-studio`. | | `googleToolSchemaPolicy?` | `"compatible" \| "reject-lossy"` | Google solamente. Omitted/ `compatible` conserva el esquema de herramienta compatible y la reparación no directa existente de 400. `reject-lossy` rechaza la pérdida inicial del esquema o una comparación limitada indeterminada antes del envío y retiene una reparación de Vertex o Cloud Code Assist que abriría restricciones. Direct AI Studio nunca realiza esa reparación. | | `directGeminiWireRenames?` | `boolean` | Google solamente. Se aplica en el modo AI Studio, incluidas las URL personalizadas compatibles con Google. Omitida o `true` mantiene el nombre del cable `-tiered` para Gemini Flash ids ( `gemini-3.7-flash` -? `gemini-3.7-flash-tiered` ); `false` envía los identificadores desnudos solicitados al cable sin cambios. Las rutas Vertex y OpenCode Zen con clave canónica conservan el ID del modelo solicitado independientemente de esta configuración; El enrutamiento de Cloud Code Assist no cambia. Establecer `false` cuando el configurado aguas arriba todavía sirve a los ID desnudos. | diff --git a/docs-site/src/content/docs/guides/claude-code.md b/docs-site/src/content/docs/guides/claude-code.md index 59477ff6b3a..461a60923ee 100644 --- a/docs-site/src/content/docs/guides/claude-code.md +++ b/docs-site/src/content/docs/guides/claude-code.md @@ -1106,6 +1106,12 @@ request and returns a `server_tool_use` block named `advisor` followed by an - A declaration whose `model` the caller's API key scope does not admit is not forwarded, and the request log records `advisor-tool-outside-model-scope`. The managed native Messages lane refuses such a request instead. +- If the request's captured policy has an applicable root, identity or provider-pool token spend + ceiling, the hosted Advisor declaration, completed history and beta are withheld. The main turn + continues. On directly routed requests the log records `advisor-tool-withheld-by-spend-policy`. + This uses the policy captured for that request, including + its selected account and pool; later configuration changes do not alter the decision. Requests + without an applicable ceiling keep the forwarding rules above. - The `server_tool_use` and `advisor_tool_result` blocks reach Claude Code unchanged, with their `srvtoolu_` ids, for streaming and non-streaming requests. That covers the plaintext `advisor_result`, the encrypted `advisor_redacted_result` and `advisor_tool_result_error`. @@ -1119,9 +1125,9 @@ request and returns a `server_tool_use` block named `advisor` followed by an declaration and the beta, so the model can consult again. - When the OpenCodex Advisor owns the name for a turn, Claude Code's earlier Advisor exchanges are also withheld for every request of that turn, so the prompt does not change partway through it. -- Anthropic reports the advisor model's tokens separately from the main model's usage, and - OpenCodex does not add them to its usage log, key spend or the usage Claude Code receives. - Check Anthropic's console for the advisor cost. +- OpenCodex does not independently account for the hosted advisor model's tokens in its usage log, + key spend or the usage Claude Code receives. This is why token ceilings withhold that optional + tool. When it is forwarded without a ceiling, check Anthropic's console for the advisor cost. ### Unicode-property patterns in tool schemas diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 5bb01b4d892..5c9fc0ef4da 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -435,7 +435,7 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity | `noVisionModels?` | `string[]` | Text-only models sent through the vision sidecar; matching tolerates an Ollama `:size` tag. | | `escapeBuiltinToolNames?` | `boolean` | Escape built-in tool names for Anthropic-compatible gateways and restore them in returned calls. | | `anthropicEofTolerance?` | `boolean` | Let an Anthropic-compatible gateway complete a stream that ends before `message_stop`, only when visible text or a complete JSON-object tool input was received. Off by default. | -| `forwardAdvisorTool?` | `boolean` | Claude Code's typed Advisor server tool (`advisor_20260301`) on the translated Messages path. Omitted: only an `anthropic`-adapter provider on first-party `https://api.anthropic.com` receives the declaration, the client's completed Advisor exchanges and the `advisor-tool-2026-03-01` beta. `true`: an Anthropic-compatible gateway that runs the Advisor receives them too. `false`: never forwarded, including first-party. Other destinations drop the declaration and the turn runs without advice. An active OpenCodex Advisor owns the `advisor` name for its turns. See [Anthropic's Advisor tool on translated routes](/guides/claude-code/#anthropics-advisor-tool-on-translated-routes). A same-destination save keeps an omitted value and a destination move drops it; `PATCH` accepts a boolean or `null` to clear it. | +| `forwardAdvisorTool?` | `boolean` | Claude Code's typed Advisor server tool (`advisor_20260301`) on the translated Messages path. Omitted: only an `anthropic`-adapter provider on first-party `https://api.anthropic.com` receives the declaration, the client's completed Advisor exchanges and the `advisor-tool-2026-03-01` beta. `true`: an Anthropic-compatible gateway that runs the Advisor receives them too. `false`: never forwarded, including first-party. Other destinations drop the declaration and the turn runs without advice. An active OpenCodex Advisor owns the `advisor` name for its turns. An applicable request token spend ceiling also withholds the declaration, completed Advisor history and beta. See [Anthropic's Advisor tool on translated routes](/guides/claude-code/#anthropics-advisor-tool-on-translated-routes). A same-destination save keeps an omitted value and a destination move drops it; `PATCH` accepts a boolean or `null` to clear it. | | `googleMode?` | `"ai-studio" \| "vertex" \| "cloud-code-assist"` | Google transport/auth mode. Default `ai-studio`. | | `googleToolSchemaPolicy?` | `"compatible" \| "reject-lossy"` | Google only. Omitted/`compatible` preserves the compatible tool schema and existing non-direct 400 repair. `reject-lossy` rejects initial schema loss or an indeterminate bounded comparison before dispatch and withholds a Vertex or Cloud Code Assist repair that would open constraints. Direct AI Studio never performs that repair. | | `directGeminiWireRenames?` | `boolean` | Google only. Applies in AI Studio mode, including custom Google-compatible URLs. Omitted or `true` keeps the `-tiered` wire rename for Gemini Flash ids (`gemini-3.7-flash` -> `gemini-3.7-flash-tiered`); `false` sends the requested bare ids to the wire unchanged. Vertex and canonical keyed OpenCode Zen routes preserve the requested model ID regardless of this setting; Cloud Code Assist routing is unchanged. Set `false` when the configured upstream still serves the bare ids. | diff --git a/src/adapters/anthropic/advisor-tool.ts b/src/adapters/anthropic/advisor-tool.ts index 4d7ee9fd941..b6a22ca3ce4 100644 --- a/src/adapters/anthropic/advisor-tool.ts +++ b/src/adapters/anthropic/advisor-tool.ts @@ -100,16 +100,26 @@ function runHasOtherServerTool(run: readonly Rec[]): boolean { return run.some(block => block.type === "server_tool_use" && block.name !== ADVISOR_SERVER_TOOL_NAME); } -/** - * Withhold the caller's Advisor for a turn the proxy Advisor joins: its declaration and every - * replayed run that contains an Advisor call. The synthetic `advisor` owns the name for the whole - * turn, so a later leg that drops the synthetic tool (consultations exhausted) must not bring the - * history or the beta back and change the prompt mid-turn. Returns whether a declaration was withheld. - */ -export function withholdAdvisorServerTools(parsed: { +type AdvisorServerToolState = { _anthropicServerTools?: readonly Rec[]; _anthropicServerReplay?: ReadonlyMap; -}): boolean { +}; + +/** A declaration or completed replay can enable the hosted Advisor on this turn. */ +export function hasAdvisorServerTools(parsed: AdvisorServerToolState): boolean { + if (hasAdvisorDeclaration(parsed._anthropicServerTools)) return true; + for (const run of parsed._anthropicServerReplay?.values() ?? []) { + if (runHasAdvisor(run)) return true; + } + return false; +} + +/** + * Withhold the caller's hosted Advisor for the whole turn: its declaration and every replayed run + * that contains an Advisor call. Later legs must not restore its history or beta, whether a spend + * ceiling withheld it or the proxy Advisor owns its name. Returns whether a declaration was withheld. + */ +export function withholdAdvisorServerTools(parsed: AdvisorServerToolState): boolean { const declared = hasAdvisorDeclaration(parsed._anthropicServerTools); if (declared) parsed._anthropicServerTools = parsed._anthropicServerTools!.filter(tool => !isAdvisorServerToolType(tool.type)); const replay = parsed._anthropicServerReplay; diff --git a/src/lib/request-execution-budget.ts b/src/lib/request-execution-budget.ts index e59591cb804..ec1aafdf8f1 100644 --- a/src/lib/request-execution-budget.ts +++ b/src/lib/request-execution-budget.ts @@ -168,6 +168,8 @@ export interface RequestSendObserver { readonly enforced?: boolean; readonly policyStarted?: boolean; readonly spendAdmissionPolicy?: SpendAdmissionPolicy; + /** Capture/query original ceilings for a selected target without starting accounting. */ + hasCapturedSpendCeiling?(target?: SpendScopes): boolean; /** Capture request policy at its first admission or actual physical start. */ startRequest?(target?: SpendScopes & { targetKey?: string }): void; ensureSeed?(target: SpendScopes & { targetKey?: string; auxiliary?: boolean; estimate?: { inputTokens: number; outputCeilingTokens: number }; explicitEstimate?: boolean }, proof?: SpendReservationProof): SpendSeed | undefined; @@ -192,6 +194,8 @@ export interface RequestExecutionBudget extends TransientSendBudget { readonly workflowCountObserved?: boolean; readonly spendPolicyStarted?: boolean; readonly spendAdmissionPolicy?: SpendAdmissionPolicy; + /** Optional hosted work checks the original ceilings even before an account was selected. */ + hasCapturedSpendCeiling?(target?: SpendScopes): boolean; startRequest?(target?: SpendScopes & { targetKey?: string }): void; claimPhysicalSend?(auxiliary?: boolean): number | undefined; /** A validated explicit successor still pays root/spend admission, not the recovery allowance. */ @@ -491,6 +495,7 @@ function createRequestExecutionBudgetWithLedger( get workflowCountObserved(): boolean { return observer?.beforePhysicalSend !== undefined; }, get spendPolicyStarted(): boolean { return observer?.policyStarted === true; }, get spendAdmissionPolicy(): SpendAdmissionPolicy | undefined { return observer?.spendAdmissionPolicy; }, + hasCapturedSpendCeiling(target) { return observer?.hasCapturedSpendCeiling?.(target) ?? observer?.enforced === true; }, startRequest(target) { observer?.startRequest?.(target); if (observer?.enforced && counter.physicalLimit === undefined) { diff --git a/src/protocols/contract.ts b/src/protocols/contract.ts index 424bf644c58..72c30e66056 100644 --- a/src/protocols/contract.ts +++ b/src/protocols/contract.ts @@ -84,6 +84,8 @@ export const PROTOCOL_REASON_CODES = [ "advisor-tool-shadowed-by-proxy-advisor", /** A typed Advisor declaration named a model outside the caller's admission scope and was not forwarded. */ "advisor-tool-outside-model-scope", + /** The request enforces token spend ceilings, so the separately unaccounted hosted Advisor was withheld. */ + "advisor-tool-withheld-by-spend-policy", ] as const; export type ProtocolReasonCode = (typeof PROTOCOL_REASON_CODES)[number]; diff --git a/src/server/responses/request-send-budget.ts b/src/server/responses/request-send-budget.ts index 68d9ef317ce..fa641fa729b 100644 --- a/src/server/responses/request-send-budget.ts +++ b/src/server/responses/request-send-budget.ts @@ -474,6 +474,7 @@ function adapterDispatchBudgetView( get spendEnforced() { return budget.spendEnforced; }, get spendPolicyStarted() { return budget.spendPolicyStarted; }, get spendAdmissionPolicy() { return budget.spendAdmissionPolicy; }, + hasCapturedSpendCeiling: target => budget.hasCapturedSpendCeiling?.(target) ?? budget.spendEnforced === true, startRequest: target => budget.startRequest?.(target), claimPhysicalSend: auxiliary => auxiliary ? budget.claimPhysicalSend?.(true) : budget.claimPhysicalSend?.(), claimContinuationPhysicalSend: () => budget.claimContinuationPhysicalSend?.(), diff --git a/src/server/responses/request-spend-target.ts b/src/server/responses/request-spend-target.ts index 4ffcee38b86..8433ed6f8b7 100644 --- a/src/server/responses/request-spend-target.ts +++ b/src/server/responses/request-spend-target.ts @@ -3,11 +3,21 @@ import type { RequestLogContext } from "../request-log"; type PhysicalSpendTarget = SpendScopes & { targetKey?: string }; const targets = new WeakMap PhysicalSpendTarget>(); +const hostedAdvisorTargets = new WeakMap PhysicalSpendTarget>(); /** A prepared candidate can own admission without overwriting the last physical send's log. */ export function bindResponsesSpendTarget(context: RequestLogContext, target?: () => PhysicalSpendTarget): void { if (target) targets.set(context, target); - else targets.delete(context); + else { targets.delete(context); hostedAdvisorTargets.delete(context); } +} + +/** Optional hosted work checks an admitted identity without changing main-model accounting. */ +export function bindResponsesHostedAdvisorSpendTarget(context: RequestLogContext, target: () => PhysicalSpendTarget): void { + hostedAdvisorTargets.set(context, target); +} + +export function responsesHostedAdvisorSpendTarget(context: RequestLogContext, fallbackPool?: string): PhysicalSpendTarget { + return hostedAdvisorTargets.get(context)?.() ?? responsesSpendTarget(context, fallbackPool); } /** Resolve late, after queued negotiation; ordinary callers retain their original pool fallback. */ diff --git a/src/server/responses/request-spend.ts b/src/server/responses/request-spend.ts index 4eade366a97..7c7816c6f21 100644 --- a/src/server/responses/request-spend.ts +++ b/src/server/responses/request-spend.ts @@ -83,6 +83,12 @@ export function createRequestSpendTracker( let ledgerRef: SpendReservationLedger | undefined = injected; const ledger = (): SpendReservationLedger => (ledgerRef ??= sharedSpendLedger()); let admissionPolicy: SpendAdmissionPolicy | undefined; + let capturedPolicy: SpendAdmissionPolicy | undefined; + const capturePolicy = (): SpendAdmissionPolicy => { + if (capturedPolicy) return capturedPolicy; + const policy = ledgerRef?.policy ?? sharedSpendPolicy(); + return capturedPolicy = Object.freeze({ root: Object.freeze({ ...policy.root }), identity: Object.freeze({ ...policy.identity }), pool: Object.freeze({ ...policy.pool }) }); + }; let initialEnforced: boolean | undefined; // Outstanding entries; exact dispatch reports move their reservation to the end. const live: string[] = []; @@ -107,13 +113,12 @@ export function createRequestSpendTracker( ...(target.identityId ?? logCtx.accountLogLabel ? { identityId: target.identityId ?? logCtx.accountLogLabel } : {}), ...(target.poolId ?? logCtx.spendPoolId ?? logCtx.provider ? { poolId: target.poolId ?? logCtx.spendPoolId ?? logCtx.provider } : {}), }); - const applicable = (scopes: SpendScopes): boolean => { - if (initialEnforced !== undefined) return initialEnforced; - const policy = admissionPolicy ?? ledgerRef?.policy ?? sharedSpendPolicy(); - return (scopes.rootId !== undefined && policy.root.maxTokens !== undefined) - || (scopes.identityId !== undefined && policy.identity.maxTokens !== undefined) - || (scopes.poolId !== undefined && policy.pool.maxTokens !== undefined); - }; + const policyApplies = (policy: SpendAdmissionPolicy, scopes: SpendScopes): boolean => + (scopes.rootId !== undefined && policy.root.maxTokens !== undefined) + || (scopes.identityId !== undefined && policy.identity.maxTokens !== undefined) + || (scopes.poolId !== undefined && policy.pool.maxTokens !== undefined); + const applicable = (scopes: SpendScopes): boolean => initialEnforced + ?? policyApplies(admissionPolicy ?? capturedPolicy ?? ledgerRef?.policy ?? sharedSpendPolicy(), scopes); const estimates = () => ({ inputTokens: logCtx.spendInputEstimateTokens ?? logCtx.usageLogInputTokens ?? 0, outputCeilingTokens: logCtx.spendOutputCeilingTokens ?? 0 }); const seedState = (seed: SpendSeed) => [...seeds.values()].find(state => state.seed === seed); @@ -249,11 +254,13 @@ export function createRequestSpendTracker( }, startRequest(target) { if (admissionPolicy) return; - const policy = ledgerRef?.policy ?? sharedSpendPolicy(); - admissionPolicy = Object.freeze({ root: Object.freeze({ ...policy.root }), identity: Object.freeze({ ...policy.identity }), pool: Object.freeze({ ...policy.pool }) }); + admissionPolicy = capturePolicy(); initialEnforced = applicable(selectedScopes(target)); if (!initialEnforced) admissionPolicy = Object.freeze({ root: Object.freeze({}), identity: Object.freeze({}), pool: Object.freeze({}) }); }, + // Combo prepayment can fix observe-only accounting before its child has an identity. + // Optional hosted work still consults the original ceilings, never the live ledger policy. + hasCapturedSpendCeiling(target) { return policyApplies(capturePolicy(), selectedScopes(target)); }, get enforced(): boolean { return applicable(selectedScopes()); }, get policyStarted(): boolean { return admissionPolicy !== undefined; }, get spendAdmissionPolicy(): SpendAdmissionPolicy | undefined { return admissionPolicy; }, diff --git a/src/server/responses/request-transport.ts b/src/server/responses/request-transport.ts index 5355903bb39..3b419fc6479 100644 --- a/src/server/responses/request-transport.ts +++ b/src/server/responses/request-transport.ts @@ -69,7 +69,8 @@ import { tryKiroAlternateAfterTerminalRefresh } from "../../oauth/kiro-terminal- import { classifyModelFamilyForQuota } from "../../oauth/account-quota-rank"; import { expandInferenceOAuthSendBudget } from "../inference/context"; import { stampOAuthAccountLabel, stampApiKeyAccountLabel, usesApiKeyAccount } from "../../providers/label"; -import { bindResponsesSpendTarget, responsesSpendTarget } from "./request-spend-target"; +import { apiKeyAccountLogLabel, oauthAccountLogLabel } from "../../codex/account-label"; +import { bindResponsesHostedAdvisorSpendTarget, bindResponsesSpendTarget, responsesSpendTarget } from "./request-spend-target"; import { isXaiResponsesDestination, resolveProviderTransport } from "../../providers/xai-transport"; import { captureRouteStaticPolicy } from "../../router"; import { CopilotAutoHttpError, resolveCopilotAuto } from "../../providers/github-copilot-auto"; @@ -151,10 +152,6 @@ export async function prepareResponsesTransport( const deferPhysicalAttribution = route.providerName === "github-copilot" && route.provider.authMode !== "forward"; let copilotCandidate: { model: string; adapter: string; accountLogLabel?: string } | undefined; bindResponsesSpendTarget(logCtx); - if (deferPhysicalAttribution) bindResponsesSpendTarget(logCtx, () => ({ - poolId: logCtx.spendPoolId ?? route.providerName, - identityId: copilotCandidate ? copilotCandidate.accountLogLabel : logCtx.accountLogLabel, - })); const genericAffinityMode = genericSessionAffinityEnabled(config, route.providerName); const anthropicInstance = route.provider.authMode === "oauth" ? configuredAnthropicInstance(config, route.providerName) : undefined; @@ -231,6 +228,18 @@ export async function prepareResponsesTransport( ? captureOAuthAccountSelection(route.providerName) : null; const anthropicManualSelectionGeneration = anthropicRouting?.captureAnthropicManualSelectionGeneration(); let servingOAuthSnapshot: OAuthAccessSnapshot | undefined; + if (deferPhysicalAttribution) bindResponsesSpendTarget(logCtx, () => ({ + poolId: logCtx.spendPoolId ?? route.providerName, + identityId: copilotCandidate ? copilotCandidate.accountLogLabel : logCtx.accountLogLabel, + })); + // Hosted-Advisor identity follows admitted credentials without changing main-model accounting. + // Anthropic's log uses a provider tag; its snapshot keeps same-ID instances distinct here. + if (usesApiKeyAccount(route.provider) || anthropicInstance) bindResponsesHostedAdvisorSpendTarget(logCtx, () => ({ + poolId: logCtx.spendPoolId ?? route.providerName, + identityId: usesApiKeyAccount(route.provider) ? apiKeyAccountLogLabel(route.providerName, route.provider._apiKeyAttempt) + : anthropicInstance && servingOAuthSnapshot ? oauthAccountLogLabel(servingOAuthSnapshot.accountId, servingOAuthSnapshot.provider) + : logCtx.accountLogLabel, + })); // These owners also serve early passthrough and sidecar sends. A dispatch-time // rebuild must update every later builder, without entering a later block's TDZ. let adapter: ProviderAdapter; diff --git a/src/server/responses/sidecar-execution.ts b/src/server/responses/sidecar-execution.ts index e8a4d7d4e9f..d8c4b46d815 100644 --- a/src/server/responses/sidecar-execution.ts +++ b/src/server/responses/sidecar-execution.ts @@ -11,8 +11,9 @@ import { formatErrorResponse } from "../../bridge"; import { planWebSearch, buildWebSearchTool, runWithWebSearch } from "../../web-search"; import { createRegisteredAdvisorPlan } from "./advisor-plan-slot"; import { ADVISOR_TOOL_NAME, createAdvisorSidecarStep } from "./advisor-slot"; -import { advisorServerToolDestination, withholdAdvisorServerTools } from "../../adapters/anthropic/advisor-tool"; +import { advisorServerToolDestination, hasAdvisorServerTools, withholdAdvisorServerTools } from "../../adapters/anthropic/advisor-tool"; import { addProtocolEntryReason } from "../../protocols/trace"; +import { responsesHostedAdvisorSpendTarget } from "./request-spend-target"; import { buildToolBridgeMaps } from "./collaboration"; import { planImageBridge, @@ -109,7 +110,7 @@ export async function executeResponsesSidecars( | "notifyResponseComplete" | "cancelResponseCompletion" >, - sendBudgetState: Pick, + sendBudgetState: Pick, ) { const { config, options, logCtx } = requestContext; const antigravityPoolActivated = requestState.route.providerName === "google-antigravity" @@ -187,6 +188,18 @@ export async function executeResponsesSidecars( } } + // Hosted Advisor usage is not independently accounted by the main model's terminal usage. + // Read the actual authenticated target's captured ceilings before an optional plan can await; + // this query never starts or changes the main-model accounting mode. + if (hasAdvisorServerTools(parsed) && advisorServerToolDestination(route.provider)) { + const budget = sendBudgetState.adapterSendBudget; + const target = responsesHostedAdvisorSpendTarget(logCtx, route.providerName); + if (budget?.hasCapturedSpendCeiling?.(target) ?? budget?.spendEnforced) { + withholdAdvisorServerTools(parsed); + addProtocolEntryReason(logCtx, "advisor-tool-withheld-by-spend-policy"); + } + } + // Advisor sidecar plan (optional subsystem; null when disabled, unconfigured, or when this // request IS an advisor loopback consultation — the recursion fence). The plan carries // request-scoped state only; preflight keeps fixed policy separate from user-role advice diff --git a/structure/advisor.md b/structure/advisor.md index ff8712c3f89..0eadc2590d3 100644 --- a/structure/advisor.md +++ b/structure/advisor.md @@ -96,12 +96,25 @@ reason `advisor-tool-shadowed-by-proxy-advisor` for a destination that would oth `planAnthropicServerTools` (`src/adapters/anthropic/advisor-tool.ts`) also withholds the completed Advisor history and the beta while a function tool named `advisor` owns the name, the same rule that keeps the web-search sidecar's `web_search` ahead of a typed search. With -this sidecar off, the declaration reaches first-party Anthropic, or a gateway with -`forwardAdvisorTool: true`, and its blocks round-trip to the client. Destination, replay and scope +this sidecar off and no applicable token spend ceiling, the declaration reaches first-party +Anthropic, or a gateway with `forwardAdvisorTool: true`, and its blocks round-trip to the client. Destination, replay and scope rules are in [providers and adapters](providers-and-adapters.md); `tests/claude-integration/claude-translated-advisor-tool.test.ts` covers both sides of the precedence through the Responses core. +Translated turns with an applicable root, identity or pool token ceiling withhold the hosted +declaration, completed Advisor history and beta, while the main model and other tools continue. +`sidecar-execution.ts` checks the actual authenticated target's captured ceilings before optional +planning without starting main-model accounting; an already captured Combo policy stays fixed. The budget's read-only +`hasCapturedSpendCeiling` checks the original numeric ceilings against that target, including an +identity selected after Combo prepayment. It does not change the initial accounting mode or reread +live config. Its hosted-only identity binding leaves ordinary accounting scopes unchanged. +An existing entry trace receives `advisor-tool-withheld-by-spend-policy`; unmarked +Combo children retain the existing trace behavior. Main-model terminal usage does not independently account for +the hosted Advisor. The proxy Advisor retains its separately budgeted consultations; see +[Responses spend](transports/responses-spend.md). The same integration test covers all three scopes, +streaming and buffered responses, replay, opted-in gateways, and policy changes after capture. + ## Recursion fence (server-owned authority) The consultation executor calls the proxy's own `/v1/chat/completions` on loopback and presents diff --git a/structure/providers-and-adapters.md b/structure/providers-and-adapters.md index 0bb1219a699..2c9edcf51dd 100644 --- a/structure/providers-and-adapters.md +++ b/structure/providers-and-adapters.md @@ -205,7 +205,7 @@ rewrite rules and the routed-id settlement. | `src/adapters/openai-responses/transport.ts` | Shared Responses URL, configured auth and caller-header construction for ordinary requests and explicitly enabled custom native v1 compact. Its compact-handoff metadata copier uses the existing client-header allowlist, never credentials, through internal request construction and Codex auth materialization. Protocol capability never grants canonical identity; v2 trigger and native blob replay remain independently declared. On the noncanonical forward path, configured `baseUrl` trailing-slash stripping is a linear scan that preserves interior slash runs and leaves the canonical forward base and non-forward auth modes untouched; `tests/responses/responses-forward-transport-baseurl.test.ts` pins it. | | `src/responses/muse-tool-name-alias.ts` | Host-gated Meta Muse 64-char tool-name alias/restore used by the Responses passthrough. | | `src/adapters/openai-chat.ts`, `src/adapters/openai-chat/` | OpenAI-compatible Chat Completions bridge, split into leaves (`wire.ts`, `messages.ts`, `response-events.ts`, `passthrough.ts`, `parallel-tool-calls.ts`, `reasoning-wire.ts`, `serialized-tool-call-content.ts`, `serialized-tool-call-markup.ts`, `standalone-bare-tool-calls.ts`, `tool-call-validation.ts`, `tool-call-id-remint.ts`, `tool-schema.ts`, `errors.ts`). `parallel-tool-calls.ts` owns the `parallel_tool_calls` wire value for both the translated and native builders, so the three provider states — configured opt-out, configured opt-in, and the unset default that forwards only a caller's explicit `false` — cannot drift between them. `reasoning-wire.ts` applies explicit gateway-object and tool-bearing effort-omission declarations to both builders; absent declarations leave native raw forwarding unchanged. Its client delivery shapes in `src/chat/outbound.ts` and `src/server/chat-native-sse.ts` relay the upstream `service_tier` echo on non-stream, folded-stream, and synthesized-SSE bodies, never inventing the key when the upstream omits it. | -| `src/adapters/anthropic.ts` | Anthropic Messages bridge. A `refusal` or `content_filter` stop reason yields an explicit `incomplete` event with `retryable: false` rather than `done` with that stopReason (#4312); `max_tokens` remains `done`. For a Responses client the bridge delivers that event, and the same one from the Kiro adapter, as a terminal `response.failed` ([Adapter-to-Responses bridge](transports/responses.md#adapter-to-responses-bridge)). It is the wire that defines `tools[*].strict` and `tools[*].allowed_callers`, so a rebuilt declaration carries both: an explicit `strict: true` and any `allowed_callers` the caller declared. An absent `strict` stays absent, because the Messages inbound records it as `false` and a `false` on the wire would read as an opt-out nobody asked for. Anthropic Fast uses the native `anthropic-speed` FastWire: a set decision sends `speed: "fast"` with `fast-mode-2026-02-01` in one case-insensitively merged, deduplicated `anthropic-beta` header that preserves OAuth betas. Stream and buffered `usage.speed` echoes confirm fast or downgrade to standard; no echo leaves the request assumed. `tests/adapters/anthropic/anthropic-fast-speed.test.ts` pins the wire and echoes. Anthropic Fast is opt-in: the registry marks both Anthropic entries `fastOptIn`, and `src/providers/fast-opt-in.ts` (`providerFastSwitchOff`) keeps Fast off until `providers..fastEnabled` is `true`. An off switch is provider capability `false`, applied in the FastPolicy authority (`service-tier.ts`), `resolveModelPolicy`, and router registry enrichment, so no model-level Fast toggle, `--fast` row, or proxy-generated `speed` field is produced. Native Claude Messages passthrough still forwards a `speed` field the caller sends itself, outside the proxy Fast policy. `tests/adapters/anthropic/anthropic-fast-opt-in.test.ts` pins the default, the switch, and the management PATCH/GET. For `api.anthropic.com` only, the Claude client's typed web tools (`parsed._anthropicServerTools`) follow the function tools unchanged, minus the caller's `cache_control`; a function tool of the same name wins, so the web-search sidecar's `web_search` keeps priority and `web_fetch` is still carried. No beta header is added: both tools are GA. Each streamed `web_search_tool_result` or `web_fetch_tool_result` becomes a `web_search_call_begin`/`web_search_call_end` pair (the query of its `server_tool_use` in the same response, hit URLs and titles, or `failed` for an error result) carrying `anthropic`: that `server_tool_use` with its complete streamed input and the result block, verbatim. An orphan result, a result with a different call id, or a result whose call input cannot be parsed is dropped with no synthetic call; the surrounding text is preserved. Streamed server-tool input retains at most 4,096 characters, slicing each incoming fragment to the remaining capacity; an oversized input that cannot be recovered drops its exchange under the same rule. The bridge and the direct encoder keep both on the `web_search_call` item as `anthropic_blocks` (a fetch is an `open_page` action), and the Claude outbound renders them back unchanged, with the original `srvtoolu_` id, `encrypted_content` and `page_age`; a fetch is not counted in `web_search_requests`. A search Anthropic defers beside a client `tool_use` has no result in that response and surfaces nothing. On the next turn, `messagesToAnthropicFormat` re-inserts the client's replayed pairs (`parsed._anthropicServerReplay`) ahead of the `tool_use`, text or thinking block that followed them, for `api.anthropic.com` only. An anchor must occur exactly once across assistant history, counting string and block content alike; repeated anchors drop their exchanges and preserve the other turns. Replay requires one-to-one call/result ids and matching tool types within the same assistant turn, with the call preceding its result in the original sequence. A single forward pass tracks pending calls and rejects reversed pairs, duplicate ids and orphan results. Calls require a plain input object with a nonempty `web_search.query` or `web_fetch.url` string; malformed input, a pair without `encrypted_content`, or a pair with no following anchor stays dropped. Stored `anthropic_blocks` restore only when call id, result id, tool/result types and input object agree; malformed metadata falls back to the rebuilt pair. Citations, a deferred search and `pause_turn` are not resumed. `tests/claude-integration/claude-translated-server-tools.test.ts` pins the wire, the sidecar priority, the bridge's client stream and JSON, and the direct encoder's stream. Claude Code's typed Advisor (`advisor_YYYYMMDD`) rides the same lane under `planAnthropicServerTools` in `src/adapters/anthropic/advisor-tool.ts`: the declaration and the client's completed Advisor exchanges go to an `anthropic`-adapter provider whose credential domain is first-party Anthropic, or to a gateway with `forwardAdvisorTool: true` (`false` opts out; the Zed delegate sets it because its wire is Zed), and only while no function tool named `advisor` (the proxy Advisor's synthetic tool) owns the name; either one adds the `advisor-tool-2026-03-01` beta. A streamed `advisor_tool_result` pairs with the same response's `server_tool_use`, whose input must be empty, and leaves as a `web_search_call_begin`/`web_search_call_end` pair with no queries whose `anthropic` carries both blocks verbatim (`failed` for `advisor_tool_result_error`, never counted in `web_search_requests`). A result whose content is not `advisor_result`, `advisor_redacted_result` or `advisor_tool_result_error` with string fields only is dropped, and the Claude outbound restores stored blocks under the same check. A call pending at `pause_turn` is not surfaced. `tests/claude-integration/claude-translated-advisor-tool.test.ts` pins the destinations, the beta, the three client lanes and replay. On translated Claude Messages requests, the inbound path carries the caller's last cache-marked system prefix out of the request body. When that prefix still matches the translated instructions after routed-model identity normalization and has a trailing system block, the Anthropic adapter splits it from the per-turn tail and moves its existing system cache breakpoint to the prefix; absent or mismatched prefixes keep the prior single-block behavior. `tests/claude-integration/claude-translated-cache-ttl.test.ts` checks the stable cached bytes across changing tails. Prompt-cache TTL: an omitted `cacheRetention` resolves through `src/adapters/anthropic/cache-retention.ts` to 1h (`ttl: "1h"`) for first-party `api.anthropic.com` (API key and OAuth pools) and to 5m for Anthropic-compatible endpoints, which do not document the 1h TTL; an explicit value wins everywhere. The direct native Messages passthrough upgrades caller breakpoints without a TTL to 1h under the same rule unless the caller pinned any breakpoint to another TTL; exact native combo replay preserves those source breakpoints unless the operator explicitly configures retention. Usage carries `cache_creation.ephemeral_1h_input_tokens` as `cacheCreation1hInputTokens`, which `src/usage/cost.ts` prices at 1.6x the 5m write rate (2x base input). `tests/adapters/adapter-usage.test.ts`, `tests/adapters/anthropic/anthropic-messages-passthrough.test.ts` and `tests/claude-integration/claude-translated-cache-ttl.test.ts` pin the defaults. | +| `src/adapters/anthropic.ts` | Anthropic Messages bridge. A `refusal` or `content_filter` stop reason yields an explicit `incomplete` event with `retryable: false` rather than `done` with that stopReason (#4312); `max_tokens` remains `done`. For a Responses client the bridge delivers that event, and the same one from the Kiro adapter, as a terminal `response.failed` ([Adapter-to-Responses bridge](transports/responses.md#adapter-to-responses-bridge)). It is the wire that defines `tools[*].strict` and `tools[*].allowed_callers`, so a rebuilt declaration carries both: an explicit `strict: true` and any `allowed_callers` the caller declared. An absent `strict` stays absent, because the Messages inbound records it as `false` and a `false` on the wire would read as an opt-out nobody asked for. Anthropic Fast uses the native `anthropic-speed` FastWire: a set decision sends `speed: "fast"` with `fast-mode-2026-02-01` in one case-insensitively merged, deduplicated `anthropic-beta` header that preserves OAuth betas. Stream and buffered `usage.speed` echoes confirm fast or downgrade to standard; no echo leaves the request assumed. `tests/adapters/anthropic/anthropic-fast-speed.test.ts` pins the wire and echoes. Anthropic Fast is opt-in: the registry marks both Anthropic entries `fastOptIn`, and `src/providers/fast-opt-in.ts` (`providerFastSwitchOff`) keeps Fast off until `providers..fastEnabled` is `true`. An off switch is provider capability `false`, applied in the FastPolicy authority (`service-tier.ts`), `resolveModelPolicy`, and router registry enrichment, so no model-level Fast toggle, `--fast` row, or proxy-generated `speed` field is produced. Native Claude Messages passthrough still forwards a `speed` field the caller sends itself, outside the proxy Fast policy. `tests/adapters/anthropic/anthropic-fast-opt-in.test.ts` pins the default, the switch, and the management PATCH/GET. For `api.anthropic.com` only, the Claude client's typed web tools (`parsed._anthropicServerTools`) follow the function tools unchanged, minus the caller's `cache_control`; a function tool of the same name wins, so the web-search sidecar's `web_search` keeps priority and `web_fetch` is still carried. No beta header is added: both tools are GA. Each streamed `web_search_tool_result` or `web_fetch_tool_result` becomes a `web_search_call_begin`/`web_search_call_end` pair (the query of its `server_tool_use` in the same response, hit URLs and titles, or `failed` for an error result) carrying `anthropic`: that `server_tool_use` with its complete streamed input and the result block, verbatim. An orphan result, a result with a different call id, or a result whose call input cannot be parsed is dropped with no synthetic call; the surrounding text is preserved. Streamed server-tool input retains at most 4,096 characters, slicing each incoming fragment to the remaining capacity; an oversized input that cannot be recovered drops its exchange under the same rule. The bridge and the direct encoder keep both on the `web_search_call` item as `anthropic_blocks` (a fetch is an `open_page` action), and the Claude outbound renders them back unchanged, with the original `srvtoolu_` id, `encrypted_content` and `page_age`; a fetch is not counted in `web_search_requests`. A search Anthropic defers beside a client `tool_use` has no result in that response and surfaces nothing. On the next turn, `messagesToAnthropicFormat` re-inserts the client's replayed pairs (`parsed._anthropicServerReplay`) ahead of the `tool_use`, text or thinking block that followed them, for `api.anthropic.com` only. An anchor must occur exactly once across assistant history, counting string and block content alike; repeated anchors drop their exchanges and preserve the other turns. Replay requires one-to-one call/result ids and matching tool types within the same assistant turn, with the call preceding its result in the original sequence. A single forward pass tracks pending calls and rejects reversed pairs, duplicate ids and orphan results. Calls require a plain input object with a nonempty `web_search.query` or `web_fetch.url` string; malformed input, a pair without `encrypted_content`, or a pair with no following anchor stays dropped. Stored `anthropic_blocks` restore only when call id, result id, tool/result types and input object agree; malformed metadata falls back to the rebuilt pair. Citations, a deferred search and `pause_turn` are not resumed. `tests/claude-integration/claude-translated-server-tools.test.ts` pins the wire, the sidecar priority, the bridge's client stream and JSON, and the direct encoder's stream. Claude Code's typed Advisor (`advisor_YYYYMMDD`) rides the same lane under `planAnthropicServerTools` in `src/adapters/anthropic/advisor-tool.ts`: the declaration and the client's completed Advisor exchanges go to an `anthropic`-adapter provider whose credential domain is first-party Anthropic, or to a gateway with `forwardAdvisorTool: true` (`false` opts out; the Zed delegate sets it because its wire is Zed), and only while no function tool named `advisor` (the proxy Advisor's synthetic tool) owns the name and the request has no applicable captured token spend ceiling ([Advisor withholding](advisor.md#claude-codes-typed-advisor-server-tool)); either declaration or replay adds the `advisor-tool-2026-03-01` beta. A streamed `advisor_tool_result` pairs with the same response's `server_tool_use`, whose input must be empty, and leaves as a `web_search_call_begin`/`web_search_call_end` pair with no queries whose `anthropic` carries both blocks verbatim (`failed` for `advisor_tool_result_error`, never counted in `web_search_requests`). A result whose content is not `advisor_result`, `advisor_redacted_result` or `advisor_tool_result_error` with string fields only is dropped, and the Claude outbound restores stored blocks under the same check. A call pending at `pause_turn` is not surfaced. `tests/claude-integration/claude-translated-advisor-tool.test.ts` pins the destinations, the beta, the three client lanes and replay. On translated Claude Messages requests, the inbound path carries the caller's last cache-marked system prefix out of the request body. When that prefix still matches the translated instructions after routed-model identity normalization and has a trailing system block, the Anthropic adapter splits it from the per-turn tail and moves its existing system cache breakpoint to the prefix; absent or mismatched prefixes keep the prior single-block behavior. `tests/claude-integration/claude-translated-cache-ttl.test.ts` checks the stable cached bytes across changing tails. Prompt-cache TTL: an omitted `cacheRetention` resolves through `src/adapters/anthropic/cache-retention.ts` to 1h (`ttl: "1h"`) for first-party `api.anthropic.com` (API key and OAuth pools) and to 5m for Anthropic-compatible endpoints, which do not document the 1h TTL; an explicit value wins everywhere. The direct native Messages passthrough upgrades caller breakpoints without a TTL to 1h under the same rule unless the caller pinned any breakpoint to another TTL; exact native combo replay preserves those source breakpoints unless the operator explicitly configures retention. Usage carries `cache_creation.ephemeral_1h_input_tokens` as `cacheCreation1hInputTokens`, which `src/usage/cost.ts` prices at 1.6x the 5m write rate (2x base input). `tests/adapters/adapter-usage.test.ts`, `tests/adapters/anthropic/anthropic-messages-passthrough.test.ts` and `tests/claude-integration/claude-translated-cache-ttl.test.ts` pin the defaults. | | `src/adapters/anthropic-model-contract.ts` | Per-family Anthropic Messages wire rules, shared by the adapter and the web-search and vision sidecars, measured live on 2026-09-29, with Haiku 5.5 documented on 2026-10-08: adaptive vs budget thinking (Haiku 5.5+ is adaptive); explicit `thinking: disabled` (Sonnet 5.0 up to but excluding 5.5, Haiku 5.5+ without effort); the `between_tools` floor that replaces it on Sonnet 5.5+ (sent without an effort, since xhigh/max reject it); forced `tool_choice` downgraded to `auto` on Opus 5.5, Fable 5.1+ and Sonnet 5.5+; `temperature`/`top_p` dropped on Opus 4.7+, Sonnet 5+, Haiku 5.5+ and every Fable, which 400 on any non-default value; `top_p` dropped when sent with `temperature` on the 4.5/4.6 families, which take either alone but not both; and sidecar thinking-off fields, a low effort with no `thinking` for Opus 5.5 and Fable, which reject both off switches. Dotted Bedrock ids do not parse as a family, and the Messages-native passthrough forwards caller fields unchanged. `tests/adapters/anthropic/anthropic-sonnet-5-5-contract.test.ts` and `tests/adapters/anthropic/anthropic-haiku-5-5-contract.test.ts` pin the family table. | | `src/adapters/google.ts` | Gemini bridge. The final wire compiler owns [endpoint-scoped tool-schema loss policy](providers/google.md#google-tool-schema-loss-reporting): compatible mode changes no request bytes, strict initial loss creates no physical send, and strict non-direct repair creates no changed repair send. A caller-declared strict tool selects `functionCallingConfig.mode: "VALIDATED"` in place of the absent-choice default; `NONE`, `ANY` and a forced-name choice are stronger constraints the caller asked for and are never overwritten. | | `src/adapters/unique-tool-call-ids.ts` | Request-scoped tool-call-id uniqueness for every `openai-chat` provider. An upstream that mints an id from the call's position in its response repeats `call-0-0` on every turn; a Messages client has already paired that id, drops the duplicate, and is left with a call that has no result, so the turn reads as empty and the model re-issues it indefinitely. Only a **repeat** is rewritten — the first occurrence stays byte-identical, leaving prompt-cache keys, reasoning-replay lookups and already-unique upstreams untouched. The ids to avoid come from the caller's history, captured in `buildRequest` (the only point that sees it) and applied at emission, never at ingestion: ingestion matches streamed deltas against the id upstream sent, so rewriting there would strip a pending call of its identity mid-stream. A repeat takes a `-` suffix, never `_`, because `_` reads as a batch sub-call of ``; occupied-set search resumes by suffix width and retained base prefix, including when siblings converge as `-9` becomes `-10`. Covered by `tests/adapters/openai/openai-chat-tool-call-id-remint.test.ts`. | diff --git a/structure/providers/anthropic-account-pool.md b/structure/providers/anthropic-account-pool.md index 327581571fd..b8b66d7494a 100644 --- a/structure/providers/anthropic-account-pool.md +++ b/structure/providers/anthropic-account-pool.md @@ -120,6 +120,10 @@ generation; native also checks its UUID. The header writer's numeric config generation is separate from the credential-generation string and preserves the existing live-roster exception. Explicit combos retain their declared targets; direct Pool 2 account recovery never selects the primary pool. +Translated Responses [hosted Advisor ceiling checks](../transports/responses-spend.md#prepaid-initial-sends) +use the admitted credential snapshot's provider and account ID, independently of display tags and +the ambient active account; equal account IDs in different instances retain separate identities. +This read-only binding does not change ordinary main-model accounting scopes. `src/oauth/anthropic-send-ownership.ts` captures the account incarnation and login identity before the physical fetch. Header publication and refusal binding diff --git a/structure/transports/responses-spend.md b/structure/transports/responses-spend.md index 97f91f0d54f..4f353ea8d22 100644 --- a/structure/transports/responses-spend.md +++ b/structure/transports/responses-spend.md @@ -110,6 +110,31 @@ cleanup releases an unsent hop once, except while an asynchronous runTurn produc Producer settlement performs that release instead, clearing only the matching pending-hop reference. Direct callers retain their prior hop reporting contract. +The optional Anthropic-hosted Advisor on translated Messages turns is withheld whenever this +request's captured policy has an applicable root, identity or pool token ceiling. Its secondary model usage +has no independent reservation and settlement contract. Before optional planning, +`src/server/responses/sidecar-execution.ts` queries the policy with the actual authenticated spend +target if a destination-eligible Advisor declaration or replay is present. The query captures original +ceilings without starting accounting, reserving a seed or choosing its mode; an already started budget +keeps its policy. `request-spend.ts` retains the original frozen ceilings even when Combo prepayment +selected observe-only accounting before its child had an identity. `hasCapturedSpendCeiling` on +`src/lib/request-execution-budget.ts` and its derived/adapter views checks that snapshot against the +actual target without changing enforcement mode, effective admission policy, physical limits or +reservations. `tests/responses/responses-first-send-spend-policy.test.ts` covers that separation and +live policy changes in both directions. The hosted-only target binding in `src/server/responses/request-transport.ts` +resolves a selected API key's existing account label before physical log attribution, so policy +checks do not record an unsent attempt as served. `request-spend-target.ts` keeps that read-only +binding separate from ordinary main-model accounting and preserves the latter's existing scopes. +For Anthropic OAuth it hashes the admitted serving snapshot's provider and account ID with the +existing OAuth label function. Same-ID accounts in the two instances stay separate; an ambient +selection change cannot replace the request's identity until the normal credential admission +accepts a new snapshot. Provider display tags and generic account log fields remain unchanged. +Both declaration and Advisor replay are removed, so neither can restore the beta. +The main turn proceeds, other tools keep their contracts, and requests without an applicable ceiling +keep destination-based forwarding. `tests/claude-integration/claude-translated-advisor-tool.test.ts` +checks the wire, unchanged main accounting, admitted identity and captured-policy changes. The separately budgeted +[proxy Advisor](../advisor.md#claude-codes-typed-advisor-server-tool) remains available. + ## Auxiliary search inference Anthropic web-search helpers in Responses, RunTurn and the passthrough bridge reserve an `auxiliary` @@ -274,13 +299,15 @@ counter, refunds decrement it, and named reports reconcile prepaid sends. Enforc the seed and physical-start protocol below across Responses, native Messages and native Chat. Each transport claims immediately before inference I/O and retains a producer through completion. -A request fixes its enforcement mode, token ceilings and physical-send limit L when it actually -starts. Configuration changes apply to requests that start afterward. An in-flight request keeps -its starting policy until completion, including when an operator enables, disables, raises or +A request fixes its enforcement mode and physical-send limit L at its first actual spend admission +or start. Numeric token ceilings are captured once: at an earlier hosted-Advisor preflight when +present, otherwise at that admission/start. Configuration changes after capture do not replace the +snapshot. An in-flight request keeps its captured policy until completion, including when an operator enables, disables, raises or lowers a ceiling. Changing policy never resets that request's physical-send allowance. A request that starts observe-only keeps the existing observe-only path throughout its lifetime. -The shared tracker captures once at the first actual spend admission or start; constructing a -budget or reading its enforcement preview does not capture policy. Generic Responses paths that +Constructing a budget or reading its ordinary enforcement preview does not capture policy. An early +hosted-Advisor ceiling query captures only the numeric snapshot; accounting mode, reservations and L +are still decided at actual admission/start. Generic Responses paths that omit retry reporters still invoke that same capture before the wire. Re-entry's token-ceiling preflight and exact seed admission use the same frozen numeric policy and prepaid proof; pool continuity validation and shared tracking-capacity safeguards remain active. diff --git a/tests/advisor/advisor-internal-authority.test.ts b/tests/advisor/advisor-internal-authority.test.ts index bd96c93c019..e9096147cdb 100644 --- a/tests/advisor/advisor-internal-authority.test.ts +++ b/tests/advisor/advisor-internal-authority.test.ts @@ -20,6 +20,8 @@ import { executeResponsesSidecars } from "../../src/server/responses/sidecar-exe import { setAdvisorPlanFactory } from "../../src/server/responses/advisor-plan-slot"; import { assertRouteAllowedByScope, resolveAdmissionModelScope } from "../../src/server/admission-model-scope"; +const workerProvider = { adapter: "openai-chat", baseUrl: "https://worker.invalid/v1", authMode: "key" } as const; + afterEach(() => { setInternalCallCapabilityForTests(null); }); @@ -50,7 +52,7 @@ describe("internal-call capability — server-owned authority", () => { const requestContext = { config, options: { admission }, logCtx: {} } as Args[0]; const requestState = { parsed: parseRequest({ model: "worker/cheap", input: [{ role: "user", content: "task" }] }), - route: { providerName: "worker", modelId: "cheap" }, admissionScope: captured, + route: { providerName: "worker", modelId: "cheap", provider: workerProvider }, admissionScope: captured, } as Args[1]; // Controlled async boundary on a prepared-state fixture. Full request // preparation/transport is intentionally not invoked by this seam test. @@ -104,7 +106,7 @@ describe("internal-call capability — server-owned authority", () => { const requestContext = { config, options, logCtx: {} } as Args[0]; const requestState = { parsed: parseRequest({ model: "worker/cheap", input: [{ role: "user", content: "task" }] }), - route: { providerName: "worker", modelId: "cheap" }, admissionScope: captured, + route: { providerName: "worker", modelId: "cheap", provider: workerProvider }, admissionScope: captured, } as Args[1]; const pending = barrier.promise.then(() => executeResponsesSidecars( requestContext, requestState, { adapter: {} } as Args[2], diff --git a/tests/claude-integration/claude-translated-advisor-tool.test.ts b/tests/claude-integration/claude-translated-advisor-tool.test.ts index 0818e73c1ed..51a4ed8c653 100644 --- a/tests/claude-integration/claude-translated-advisor-tool.test.ts +++ b/tests/claude-integration/claude-translated-advisor-tool.test.ts @@ -1,19 +1,22 @@ import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test"; -import { mkdtempSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; import { declaredAnthropicServerTools, hasAnthropicOnlyServerTool, hasNativeOnlyTool, replayedAnthropicServerBlocks } from "../../src/claude/inbound-content-options"; import { clearComboSelectionState, clearComboTargetCooldowns } from "../../src/combos"; import { clearKeyCooldowns } from "../../src/providers/key-failover"; import { markProtocolEntry, protocolTraceForRequest } from "../../src/protocols/trace"; import { handleClaudeMessages } from "../../src/server/claude-messages"; import { getRequestLogEntries } from "../../src/server/request-log"; +import type { RequestLogContext } from "../../src/server/request-log"; +import { configureSharedSpendLedger, sharedSpendLedger, spendPolicyFromConfig } from "../../src/lib/spend-reservation-ledger"; import { setAdvisorPlanFactory } from "../../src/server/responses/advisor-plan-slot"; import { clearComboRecallForTests } from "../../src/server/responses/combo-session-recall"; import { handleResponses } from "../../src/server/responses/core"; +import { responsesHostedAdvisorSpendTarget, responsesSpendTarget } from "../../src/server/responses/request-spend-target"; +import type { RequestSpendTracker } from "../../src/server/responses/request-spend"; +import type { AnthropicInstanceId } from "../../src/providers/anthropic-instance-id"; import type { OcxConfig } from "../../src/types"; import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; -import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { createTempHome } from "../helpers/temp-home"; +import { createAnthropicInstanceFixture, type AnthropicInstanceFixture } from "../helpers/anthropic-instance-fixture"; import { planAnthropicServerTools, withholdAdvisorServerTools } from "../../src/adapters/anthropic/advisor-tool"; import { webSearchPairFromItem } from "../../src/claude/outbound"; @@ -32,20 +35,15 @@ const PLAIN = { type: "advisor_result", text: "Close the input channel first.", const REDACTED = { type: "advisor_redacted_result", encrypted_content: CIPHERTEXT }; const ERRORED = { type: "advisor_tool_result_error", error_code: "overloaded" }; -let previousHome: string | undefined; -let home = ""; +let home: ReturnType; let releaseSpend: (() => void) | undefined; beforeAll(() => { - previousHome = process.env.OPENCODEX_HOME; - home = mkdtempSync(join(tmpdir(), "ocx-advisor-server-tool-")); - process.env.OPENCODEX_HOME = home; + home = createTempHome("ocx-advisor-server-tool-"); releaseSpend = acquireOwnedSpendHome(); }); afterAll(() => { releaseSpend?.(); - if (previousHome === undefined) delete process.env.OPENCODEX_HOME; - else process.env.OPENCODEX_HOME = previousHome; - removeTreeWithRetry(home); + home.remove(); }); // A refused child cools its combo target down; every case starts from a fresh waterfall. beforeEach(() => { @@ -82,9 +80,10 @@ interface Wire { body: Rec; headers: Headers } interface TurnOptions { baseUrl?: string; provider?: Rec; model?: string; stream?: boolean; direct?: boolean; messages?: unknown[]; tools?: unknown[]; frames?: Rec[]; gptFirst?: boolean; gptOnly?: boolean; nativeRollout?: boolean; requestId?: string; + spend?: OcxConfig["spend"]; rootId?: string; policyDuringPlan?: OcxConfig["spend"]; proxyAdvisor?: boolean; } -async function claudeTurn(options: TurnOptions = {}): Promise<{ wires: Wire[]; status: number; body: string }> { +async function claudeTurn(options: TurnOptions = {}): Promise<{ wires: Wire[]; status: number; body: string; logCtx: RequestLogContext }> { const wires: Wire[] = []; const capture = (target: "anth" | "gpt") => (async (_input: unknown, init?: RequestInit) => { wires.push({ body: JSON.parse(String(init?.body)) as Rec, headers: new Headers(init?.headers) }); @@ -98,6 +97,7 @@ async function claudeTurn(options: TurnOptions = {}): Promise<{ wires: Wire[]; s const gpt = { provider: "gpt", model: "g" }; const config = { port: 0, defaultProvider: "anth", + spend: options.spend, claudeCode: { nativePassthrough: false }, webSearchSidecar: { enabled: false }, ...(options.direct || options.nativeRollout @@ -108,16 +108,31 @@ async function claudeTurn(options: TurnOptions = {}): Promise<{ wires: Wire[]; s }, combos: { waterfall: { strategy: "failover", targets: options.gptOnly ? [gpt] : options.gptFirst ? [gpt, anth] : [anth, gpt] } }, } as unknown as OcxConfig; - const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { - method: "POST", - headers: { "content-type": "application/json", "anthropic-beta": BETA }, - body: JSON.stringify({ - model: options.model ?? (options.direct ? `anth/${MODEL}` : "combo/waterfall"), max_tokens: 64, stream: options.stream ?? true, - tools: options.tools ?? [READ, ADVISOR], - messages: options.messages ?? [{ role: "user", content: "Plan the shutdown." }], - }), - }), config, { model: "", provider: "" }, options.requestId ? { requestId: options.requestId, start: Date.now() } : undefined); - return { wires, status: response.status, body: await response.text() }; + configureSharedSpendLedger(spendPolicyFromConfig(config.spend)); + if (options.proxyAdvisor || options.policyDuringPlan !== undefined) setAdvisorPlanFactory(config, async () => { + if (options.policyDuringPlan !== undefined) { + config.spend = options.policyDuringPlan; + configureSharedSpendLedger(spendPolicyFromConfig(config.spend)); + } + return options.proxyAdvisor ? { + tool: { name: "advisor", description: "proxy advisor", parameters: { type: "object", properties: {} }, advisor: true }, + preflightInject: async () => false, attachGuard: () => {}, + } : null; + }); + const logCtx: RequestLogContext = { model: "", provider: "" }; + try { + const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { + method: "POST", + headers: { "content-type": "application/json", "anthropic-beta": BETA, + ...(options.rootId ? { "x-codex-parent-thread-id": options.rootId } : {}) }, + body: JSON.stringify({ + model: options.model ?? (options.direct ? `anth/${MODEL}` : "combo/waterfall"), max_tokens: 64, stream: options.stream ?? true, + tools: options.tools ?? [READ, ADVISOR], + messages: options.messages ?? [{ role: "user", content: "Plan the shutdown." }], + }), + }), config, logCtx, { requestId: options.requestId ?? `advisor-turn-${crypto.randomUUID()}`, start: Date.now() }); + return { wires, status: response.status, body: await response.text(), logCtx }; + } finally { setAdvisorPlanFactory(config, null); } } const toolTypes = (wire: Wire) => ((wire.body.tools ?? []) as Rec[]).map(tool => String(tool.type ?? "function")); @@ -143,6 +158,104 @@ const followUp = (assistant: unknown[]) => [ { role: "user", content: "Plan the shutdown." }, { role: "assistant", content: assistant }, { role: "user", content: "Go on." }, ]; +describe("hosted Advisor under the request's token spend policy", () => { + const limit = { maxTokens: 100_000 }; + const replay = followUp([ + { type: "server_tool_use", id: "srvtoolu_adv1", name: "advisor", input: {} }, + { type: "advisor_tool_result", tool_use_id: "srvtoolu_adv1", content: REDACTED }, + { type: "text", text: "Earlier answer." }, + ]); + for (const scope of ["root", "identity", "pool"] as const) for (const stream of [false, true]) { + test(`${scope} ceiling withholds the optional declaration, replay and beta (stream=${stream})`, async () => { + const rootId = `advisor-spend-${scope}-${stream}`; + const beforePool = sharedSpendLedger().snapshot("pool", "anth")?.settled ?? 0; + const result = await claudeTurn({ model: `anth/${MODEL}`, spend: { [scope]: limit }, rootId, stream, + provider: { apiKey: `sk-ant-fixture-spend-${scope}-${stream}` }, messages: replay, + tools: [READ, ADVISOR, { type: "web_search_20250305", name: "web_search" }] }); + expect(result.status, result.body).toBe(200); + expect(result.body).toContain("ok"); + expect(result.wires).toHaveLength(1); + const wire = result.wires[0]!; + expect(toolTypes(wire)).toEqual(["function", "web_search_20250305"]); + expect(betas(wire)).not.toContain(BETA); + expect(JSON.stringify(wire.body)).not.toContain("srvtoolu_adv1"); + expect(JSON.stringify(wire.body)).not.toContain(CIPHERTEXT); + expect(JSON.stringify(wire.body)).toContain("Earlier answer."); + expect(protocolTraceForRequest(result.logCtx, result.logCtx.attempts)?.reasonCodes) + .toContain("advisor-tool-withheld-by-spend-policy"); + // Read the actual provider credential identity assigned by auth; never invent a main label. + expect(result.logCtx.accountLogLabel).toBeString(); + expect(result.logCtx.accountLogLabel).not.toBe("main"); + const id = scope === "root" ? rootId : scope === "identity" ? result.logCtx.accountLogLabel! : "anth"; + const settled = sharedSpendLedger().snapshot(scope, id); + expect(settled).toBeDefined(); + expect(settled!.reserved).toBe(0); + expect(settled!.unresolved).toBe(0); + expect(settled!.settled - (scope === "pool" ? beforePool : 0)).toBe(6); + }); + } + + for (const stream of [false, true]) test(`an identity ceiling captured before Combo account selection withholds Advisor (stream=${stream})`, async () => { + const before = sharedSpendLedger().snapshot("pool", "anth")?.settled ?? 0; + const result = await claudeTurn({ model: "combo/waterfall", spend: { identity: limit }, stream, messages: replay }); + expect(result.status, result.body).toBe(200); + expect(result.body).toContain("ok"); + expect(result.wires).toHaveLength(1); + expect(result.logCtx.accountLogLabel).toMatch(/^k[a-f0-9]{32}$/); + expect(toolTypes(result.wires[0]!)).toEqual(["function"]); + expect(betas(result.wires[0]!)).not.toContain(BETA); + expect(JSON.stringify(result.wires[0]!.body)).not.toContain("srvtoolu_adv1"); + // Advisor withholding does not change the Combo's existing main-send accounting mode. + const settled = sharedSpendLedger().snapshot("pool", "anth")!; + expect([settled.settled - before, settled.reserved, settled.unresolved]).toEqual([6, 0, 0]); + }); + + test("an opted-in gateway's direct encoder withholds Advisor while keeping the main turn", async () => { + const result = await claudeTurn({ baseUrl: GATEWAY, provider: { forwardAdvisorTool: true }, direct: true, + spend: { pool: limit }, messages: replay }); + expect(result.status, result.body).toBe(200); + expect(toolTypes(result.wires[0]!)).toEqual(["function"]); + expect(betas(result.wires[0]!)).not.toContain(BETA); + expect(JSON.stringify(result.wires[0]!.body)).not.toContain(CIPHERTEXT); + }); + + test("history alone cannot restore the Advisor beta under an enforced ceiling", async () => { + const result = await claudeTurn({ model: `anth/${MODEL}`, spend: { pool: limit }, messages: replay, tools: [READ] }); + expect(result.status, result.body).toBe(200); + expect(betas(result.wires[0]!)).not.toContain(BETA); + expect(JSON.stringify(result.wires[0]!.body)).not.toContain("srvtoolu_adv1"); + expect(protocolTraceForRequest(result.logCtx, result.logCtx.attempts)?.reasonCodes) + .toContain("advisor-tool-withheld-by-spend-policy"); + }); + + test("a configured root ceiling without a root is not applicable", async () => { + const result = await claudeTurn({ model: `anth/${MODEL}`, spend: { root: limit } }); + expect(result.status, result.body).toBe(200); + expect(toolTypes(result.wires[0]!)).toContain("advisor_20260301"); + expect(betas(result.wires[0]!)).toContain(BETA); + }); + + for (const scope of ["identity", "pool"] as const) for (const initialLimited of [false, true]) for (const combo of [false, true]) { + test(`captured ${scope} ceiling survives a live change (limited=${initialLimited}, combo=${combo})`, async () => { + const result = await claudeTurn({ model: combo ? "combo/waterfall" : `anth/${MODEL}`, + spend: initialLimited ? { [scope]: limit } : {}, policyDuringPlan: initialLimited ? {} : { [scope]: limit } }); + expect(result.status, result.body).toBe(200); + expect(toolTypes(result.wires[0]!).includes("advisor_20260301")).toBe(!initialLimited); + expect(betas(result.wires[0]!).includes(BETA)).toBe(!initialLimited); + }); + } + + test("the independently budgeted proxy Advisor keeps its synthetic tool", async () => { + const result = await claudeTurn({ model: `anth/${MODEL}`, spend: { pool: limit }, proxyAdvisor: true }); + expect(result.status, result.body).toBe(200); + const advisors = (result.wires[0]!.body.tools as Rec[]).filter(tool => tool.name === "advisor"); + expect(advisors).toHaveLength(1); + expect(advisors[0]).toHaveProperty("input_schema"); + expect(toolTypes(result.wires[0]!)).not.toContain("advisor_20260301"); + expect(betas(result.wires[0]!)).not.toContain(BETA); + }); +}); + describe("advisor declarations on the translated Messages lane", () => { test("module: advisor is carried out of body and is not a native-only or Anthropic-only tool", () => { expect(declaredAnthropicServerTools({ tools: [READ, ADVISOR] })).toEqual([ADVISOR]); @@ -448,3 +561,148 @@ describe("advisor exchanges only come from, and replay with, checked fields", () ]]); }); }); + +async function withAdvisorOAuthFixture(poolEnabled: boolean, run: (f: AnthropicInstanceFixture) => Promise): Promise { + // The process can own only one spend home. Release the ordinary fixture before the isolated + // OAuth fixture takes its real lease, and reacquire only after its environment is restored. + releaseSpend?.(); releaseSpend = undefined; + let f: AnthropicInstanceFixture | undefined, release: (() => void) | undefined; + try { + f = await createAnthropicInstanceFixture({ anthropic: { enabled: poolEnabled }, anthropic2: { enabled: poolEnabled } }); + await f.seed(); + release = acquireOwnedSpendHome(); + f.config.claudeCode = { nativePassthrough: false }; + f.config.webSearchSidecar = { enabled: false }; + f.config.protocols = { rollout: { managedMessagesNative: false, managedMessagesNativeOAuth: false } }; + await run(f); + } finally { + release?.(); + try { await f?.dispose(); } + finally { releaseSpend = acquireOwnedSpendHome(); } + } +} + +async function oauthAdvisorTurn(f: AnthropicInstanceFixture, instance: AnthropicInstanceId, combo = false, + expectedSlot = 1, duringPlan?: (log: RequestLogContext) => Promise, hostedAdvisor = true, ingress: "messages" | "responses" = "messages") { + const wires: Wire[] = []; + const logCtx: RequestLogContext = { provider: "", model: "" }; + Object.assign(f.config.providers[instance]!, { models: [f.model], fetch: (async (_input: unknown, init?: RequestInit) => { + const headers = new Headers(init?.headers); + f.ledger.record({ instance, accountId: f.ids[expectedSlot - 1]!, token: headers.get("authorization")?.replace(/^Bearer /, "") ?? "" }); + const body = JSON.parse(String(init?.body)) as Rec; + wires.push({ body, headers }); + if (body.stream === false) return Response.json({ id: "msg_fixture", type: "message", role: "assistant", model: f.model, + content: [{ type: "text", text: "ok" }], stop_reason: "end_turn", usage: { input_tokens: 1, output_tokens: 5 } }); + return sse([{ ...START, message: { ...START.message, model: f.model } }, ...text(0, "ok"), ...end()]); + }) as typeof fetch }); + f.config.combos = { waterfall: { strategy: "failover", targets: [{ provider: instance, model: f.model }] } }; + configureSharedSpendLedger(spendPolicyFromConfig(f.config.spend)); + if (duringPlan) setAdvisorPlanFactory(f.config, async () => { await duringPlan(logCtx); return null; }); + try { + const model = combo ? "combo/waterfall" : `${instance}/${f.model}`; + const req = new Request(`http://localhost/v1/${ingress}`, { + method: "POST", headers: { "content-type": "application/json", ...(hostedAdvisor ? { "anthropic-beta": BETA } : {}) }, + body: JSON.stringify(ingress === "responses" ? { model, max_output_tokens: 64, stream: false, + tools: [{ type: "function", name: READ.name, description: READ.description, parameters: READ.input_schema }], input: "ordinary accounting control" } + : { model, max_tokens: 64, stream: false, + tools: hostedAdvisor ? [READ, ADVISOR] : [READ], messages: hostedAdvisor ? followUp([ + { type: "server_tool_use", id: "srvtoolu_adv1", name: "advisor", input: {} }, + { type: "advisor_tool_result", tool_use_id: "srvtoolu_adv1", content: REDACTED }, + { type: "text", text: "Earlier answer." }, + ]) : [{ role: "user", content: "ordinary accounting control" }] }), + }); + const context = { requestId: `oauth-advisor-${crypto.randomUUID()}`, start: Date.now() }; + const response = ingress === "responses" ? await handleResponses(req, f.config, logCtx, context) + : await handleClaudeMessages(req, f.config, logCtx, context); + return { status: response.status, body: await response.text(), wires, logCtx }; + } finally { setAdvisorPlanFactory(f.config, null); } +} + +describe("hosted Advisor with admitted Anthropic OAuth identities", () => { + for (const ingress of ["messages", "responses"] as const) for (const instance of ["anthropic", "anthropic2"] as const) + for (const poolEnabled of [false, true]) test(`${ingress}: ${instance}, pool=${poolEnabled} keeps ordinary accounting without hosted Advisor`, async () => { + await withAdvisorOAuthFixture(poolEnabled, async f => { + f.config.spend = { identity: { maxTokens: 1 } }; + const result = await oauthAdvisorTurn(f, instance, false, 1, undefined, false, ingress); + expect(result.status, result.body).toBe(200); + expect(result.wires).toHaveLength(1); + expect(toolTypes(result.wires[0]!)).toEqual(["function"]); + expect(betas(result.wires[0]!)).not.toContain(BETA); + expect(result.logCtx.accountLogLabel).toBeUndefined(); + const tracker = result.logCtx.spendTracker as RequestSpendTracker; + expect(tracker.policyStarted).toBe(true); + expect(tracker.enforced).toBe(false); + expect(tracker.spendAdmissionPolicy).toEqual({ root: {}, identity: {}, pool: {} }); + expect(responsesSpendTarget(result.logCtx, instance).identityId).toBeUndefined(); + const identity = responsesHostedAdvisorSpendTarget(result.logCtx, instance).identityId; + expect(identity).toMatch(/^o[a-f0-9]{6}$/); + expect(sharedSpendLedger().snapshot("identity", identity!)).toBeUndefined(); + f.ledger.assertNoCrossSend(); + }); + }); + for (const instance of ["anthropic", "anthropic2"] as const) for (const poolEnabled of [false, true]) + for (const combo of [false, true]) for (const maxTokens of [undefined, 1, 100_000]) { + test(`${instance}, pool=${poolEnabled}, combo=${combo}, identity ceiling=${maxTokens}`, async () => { + await withAdvisorOAuthFixture(poolEnabled, async f => { + const limited = maxTokens !== undefined; + f.config.spend = limited ? { identity: { maxTokens } } : {}; + const result = await oauthAdvisorTurn(f, instance, combo); + expect(result.status, result.body).toBe(200); + expect(result.body).toContain("ok"); + expect(result.wires).toHaveLength(1); + expect(toolTypes(result.wires[0]!).includes("advisor_20260301")).toBe(!limited); + expect(betas(result.wires[0]!).includes(BETA)).toBe(!limited); + expect(JSON.stringify(result.wires[0]!.body).includes("srvtoolu_adv1")).toBe(!limited); + expect(JSON.stringify(result.wires[0]!.body)).toContain("Earlier answer."); + expect(result.logCtx.accountLogLabel).toBeUndefined(); // Keep existing OAuth log attribution. + expect((result.logCtx.spendTracker as RequestSpendTracker).enforced).toBe(false); + f.ledger.assertNoCrossSend(); + if (limited && !combo) { + const identity = responsesHostedAdvisorSpendTarget(result.logCtx, instance).identityId; + expect(identity).toMatch(/^o[a-f0-9]{6}$/); + // The hosted-only identity query does not add main-model identity accounting. + expect(responsesSpendTarget(result.logCtx, instance).identityId).toBeUndefined(); + expect(sharedSpendLedger().snapshot("identity", identity!)).toBeUndefined(); + } + }); + }); + } + + test("ordinary API-key accounting keeps its existing mode under a tiny identity ceiling", async () => { + const result = await claudeTurn({ model: `anth/${MODEL}`, stream: false, spend: { identity: { maxTokens: 1 } }, + provider: { apiKey: "fixture-ordinary-identity-key" }, tools: [READ] }); + expect(result.status, result.body).toBe(200); + expect(result.wires).toHaveLength(1); + expect(toolTypes(result.wires[0]!)).toEqual(["function"]); + expect(result.logCtx.accountLogLabel).toMatch(/^k[a-f0-9]{32}$/); + const tracker = result.logCtx.spendTracker as RequestSpendTracker; + expect(tracker.policyStarted).toBe(true); + expect(tracker.enforced).toBe(false); + expect(sharedSpendLedger().snapshot("identity", result.logCtx.accountLogLabel!)).toBeUndefined(); + }); + + test("target identity belongs to the admitted snapshot and keeps same-ID instances separate", async () => { + await withAdvisorOAuthFixture(false, async f => { + f.config.spend = { identity: { maxTokens: 100_000 } }; + const first = await oauthAdvisorTurn(f, "anthropic"); + expect(first.status, first.body).toBe(200); + const firstIdentity = responsesHostedAdvisorSpendTarget(first.logCtx, "anthropic").identityId; + let selectedIdentity: string | undefined; + const second = await oauthAdvisorTurn(f, "anthropic2", false, 2, async log => { + selectedIdentity = responsesHostedAdvisorSpendTarget(log, "anthropic2").identityId; + expect(selectedIdentity).toMatch(/^o[a-f0-9]{6}$/); + expect(selectedIdentity).not.toBe(firstIdentity); + await f.store.setActiveAccount("anthropic2", f.ids[1]); + // A new ambient active account cannot impersonate this request's admitted credential. + expect(responsesHostedAdvisorSpendTarget(log, "anthropic2").identityId).toBe(selectedIdentity); + expect(log.accountLogLabel).toBeUndefined(); + }); + expect(second.status, second.body).toBe(200); + expect(second.wires).toHaveLength(1); + expect(responsesHostedAdvisorSpendTarget(second.logCtx, "anthropic2").identityId).not.toBe(selectedIdentity); + expect(toolTypes(second.wires[0]!)).toEqual(["function"]); + expect(f.ledger.sends.map(send => [send.instance, send.accountId])) + .toEqual([["anthropic", f.ids[0]], ["anthropic2", f.ids[1]]]); + }); + }); +}); diff --git a/tests/responses/responses-first-send-spend-policy.test.ts b/tests/responses/responses-first-send-spend-policy.test.ts index 8890891c6a3..69a86b1b7ef 100644 --- a/tests/responses/responses-first-send-spend-policy.test.ts +++ b/tests/responses/responses-first-send-spend-policy.test.ts @@ -7,7 +7,7 @@ import { closeRequestHistoryIndex } from "../../src/routing/history/indexer"; import { clearResponseStateForTests, flushResponseState } from "../../src/responses/state"; import { clearKeyCooldowns } from "../../src/providers/key-failover"; import { resetProviderRequestPacingForTest } from "../../src/providers/request-pacing"; -import { createPhysicalSendReporter, createRequestExecutionBudget } from "../../src/lib/request-execution-budget"; +import { createPhysicalSendReporter, createRequestExecutionBudget, deriveRequestExecutionBudget } from "../../src/lib/request-execution-budget"; import { createSpendReservationLedger, DEFAULT_SPEND_RESERVATION_POLICY, type SpendJournal, type SpendReservationPolicy } from "../../src/lib/spend-reservation-ledger"; import { SendBudgetExhaustedError, providerErrorDestinationLabel, fetchWithTransientRetry } from "../../src/lib/upstream-retry"; import { createTranslatorBudget } from "../../src/lib/translator-budget"; @@ -86,6 +86,68 @@ for (const enforced of [false, true]) test(`receipt-versus-reporter choice is fi } finally { translatorBudget.dispose(); } }); +for (const limited of [false, true]) test(`an early optional-tool query captures policy without starting accounting (limited=${limited})`, () => { + const identityPolicy = (enabled: boolean): SpendReservationPolicy => ({ + ...DEFAULT_SPEND_RESERVATION_POLICY, identity: enabled ? { maxTokens: 1_000_000 } : {}, + }); + const ledger = createSpendReservationLedger({ journal: memoryJournal(), salt: "early-optional-policy", policy: identityPolicy(limited) }); + const tracker = createRequestSpendTracker({ provider: "a", model: "m" }, undefined, ledger); + const parent = createRequestExecutionBudget(undefined, undefined, tracker); + const child = deriveRequestExecutionBudget(parent, parent.policy); + const target = { poolId: "a", identityId: "selected-fixture-key" }; + for (const budget of [parent, child]) { + expect(budget.hasCapturedSpendCeiling?.(target)).toBe(limited); + expect(budget.spendPolicyStarted).toBe(false); + expect(budget.spendAdmissionPolicy).toBeUndefined(); + expect(budget.physicalStarted).toBe(0); + expect(budget.used).toBe(0); + expect(ledger.snapshot("identity", target.identityId)).toBeUndefined(); + } + ledger.reconfigure(identityPolicy(!limited)); + expect(child.hasCapturedSpendCeiling?.(target)).toBe(limited); + // Only the real accounting admission chooses its mode, using the original policy. + parent.startRequest?.(target); + expect(parent.spendPolicyStarted).toBe(true); + expect(parent.spendEnforced).toBe(limited); + expect(parent.hasCapturedSpendCeiling?.(target)).toBe(limited); + expect(parent.physicalStarted).toBe(0); + expect(parent.used).toBe(0); +}); + +for (const limited of [false, true]) test(`optional-tool ceiling query retains captured identity policy without changing accounting (limited=${limited})`, () => { + const identityPolicy = (enabled: boolean): SpendReservationPolicy => ({ + ...DEFAULT_SPEND_RESERVATION_POLICY, identity: enabled ? { maxTokens: 1_000_000 } : {}, + }); + const ledger = createSpendReservationLedger({ journal: memoryJournal(), salt: "first-send-identity-policy", policy: identityPolicy(limited) }); + const log: RequestLogContext = { model: "m", provider: "a", spendPoolId: "a", spendOutputCeilingTokens: 64 }; + const tracker = createRequestSpendTracker(log, undefined, ledger); + const parent = createRequestExecutionBudget(undefined, undefined, tracker); + const booking = parent.reserveDispatch({ sendClass: "initial", targetKey: "a/m", countedExternally: true }); + if (!booking.allowed) throw new Error("fixture reservation refused"); + const translatorBudget = createTranslatorBudget(); + try { + const child = deriveRequestExecutionBudget(parent, parent.policy); + const owner = createResponsesSendBudget({ req: new Request("http://localhost/v1/responses"), logCtx: log, + options: { translatorBudget, sendBudget: child, comboInitialSend: { permit: booking.permit } } }); + if (owner instanceof Response) throw new Error("unexpected workflow refusal"); + const frozenAccountingPolicy = parent.spendAdmissionPolicy; + expect(frozenAccountingPolicy).toEqual({ root: {}, identity: {}, pool: {} }); + expect(parent.spendEnforced).toBe(false); // The parent did not yet have an account. + ledger.reconfigure(identityPolicy(!limited)); + const target = { poolId: "a", identityId: "selected-fixture-key" }; + for (const budget of [parent, child, owner.adapterDispatchBudget!]) { + expect(budget.hasCapturedSpendCeiling?.(target)).toBe(limited); + expect(budget.hasCapturedSpendCeiling?.({ poolId: "a" })).toBe(false); + budget.startRequest?.(target); + expect(budget.spendAdmissionPolicy).toBe(frozenAccountingPolicy); + expect(budget.spendEnforced).toBe(false); + expect(budget.physicalLimit).toBeUndefined(); + expect(budget.used).toBe(1); + } + } finally { booking.permit.release(); translatorBudget.dispose(); } + expect(parent.used).toBe(0); +}); + /** Combo -> Responses passthrough upstream, observing whether the policy had started at the wire. */ for (const enforced of [false, true]) test(`passthrough Combo send starts spend policy from its booking before the wire (enforced=${enforced})`, async () => { const f = trackerFixture(enforced); diff --git a/tests/responses/responses-spend-anchor.test.ts b/tests/responses/responses-spend-anchor.test.ts index 917d23b2e38..48a994d4b21 100644 --- a/tests/responses/responses-spend-anchor.test.ts +++ b/tests/responses/responses-spend-anchor.test.ts @@ -22,6 +22,7 @@ import { resetProviderRequestPacingForTest } from "../../src/providers/request-p import { acquireSpendLedgerServerLifecycle } from "../../src/server/index/spend-ledger-lifecycle"; import { createResponsesSendBudget } from "../../src/server/responses/request-send-budget"; import { SpendLedgerOwnerError } from "../../src/lib/spend-ledger-owner"; +import { apiKeyAccountLogLabel } from "../../src/codex/account-label"; function fixture(enforced = true, capacity = 8) { const lines: string[] = []; @@ -678,11 +679,30 @@ for (const adapter of ["azure", "azure-openai"] as const) { test("Combo fallback recomputes spend from the next target's original cap", async () => withExecutorHome(async ledger => { - ledger.reconfigure({ ...DEFAULT_SPEND_RESERVATION_POLICY, pool: { maxTokens: 128 } }); + ledger.reconfigure({ ...DEFAULT_SPEND_RESERVATION_POLICY, canonicalProviderIds: ["P", "Q"], pool: { maxTokens: 128 } }); const observations: Array<{ pool: string; cap: number }> = []; - const bookings = spyOn(ledger, "reserveSeed"); + const qBookings: Array<{ request: Parameters[0]; decision: ReturnType }> = []; + const reserveSeed = ledger.reserveSeed.bind(ledger); + const bookings = spyOn(ledger, "reserveSeed").mockImplementation((request, owner) => { + const decision = reserveSeed(request, owner); + if (request.scopes.poolId === "Q") qBookings.push({ request, decision }); + return decision; + }); + const abandons = spyOn(ledger, "abandon"); + const qIdentity = apiKeyAccountLogLabel("Q", { reference: "fixture-only" })!; const wireFor = (pool: string, fail: boolean) => (async (_url: unknown, init?: RequestInit) => { observations.push({ pool, cap: JSON.parse(String(init?.body)).max_output_tokens }); + if (pool === "Q") { + expect(qBookings).toHaveLength(1); + const [provisional] = qBookings; + expect(provisional!.request.scopes.identityId).toBeUndefined(); + expect(provisional!.decision.reserved).toBe(true); + if (!provisional!.decision.reserved) throw new Error("Q provisional reservation was refused"); + // Optional hosted-tool identity lookup must not rebind ordinary main-model accounting. + expect(abandons).not.toHaveBeenCalledWith(provisional!.decision.seed.sendId); + expect(ledger.snapshot("pool", "Q")).toMatchObject({ reserved: 1, settled: 0, unresolved: 0 }); + expect(ledger.snapshot("identity", qIdentity)).toBeUndefined(); + } return fail ? Response.json({ error: { message: "fixture overload" } }, { status: 503 }) : executorSuccess("passthrough"); }) as typeof fetch; const config = executorConfig("passthrough", wireFor("P", true)); @@ -708,10 +728,13 @@ test("Combo fallback recomputes spend from the next target's original cap", asyn { pool: "P", cap: 16 }, { pool: "Q", cap: 1 }, ]); - expect(bookings.mock.calls.filter(([request]) => request.scopes.poolId === "Q") - .map(([request]) => request.outputCeilingTokens)).toEqual([1]); + expect(qBookings.map(({ request }) => request.outputCeilingTokens)).toEqual([1]); expect(budget.physicalStarted).toBe(4); - tracker.settle(undefined); + expect(budget.used).toBe(4); + tracker.settle({ inputTokens: 2, outputTokens: 1 }); await ledger.waitForReporterDrain(); - } finally { bookings.mockRestore(); translatorBudget.dispose(); } + expect(ledger.snapshot("pool", "P")).toMatchObject({ reserved: 0, settled: 0, unresolved: 48 }); + expect(ledger.snapshot("pool", "Q")).toMatchObject({ reserved: 0, settled: 3, unresolved: 0 }); + expect(ledger.snapshot("identity", qIdentity)).toBeUndefined(); + } finally { bookings.mockRestore(); abandons.mockRestore(); translatorBudget.dispose(); } }), 10_000); diff --git a/tests/responses/sse-untyped-credential-mask.test.ts b/tests/responses/sse-untyped-credential-mask.test.ts index f902ff185f4..982f0d28fa6 100644 --- a/tests/responses/sse-untyped-credential-mask.test.ts +++ b/tests/responses/sse-untyped-credential-mask.test.ts @@ -1,9 +1,11 @@ -import { describe, expect, test } from "bun:test"; +import { describe, expect, spyOn, test } from "bun:test"; import { addFinalRequestLog, inspectResponseLogSsePayload, type RequestLogContext } from "../../src/server/request-log"; import { relaySseWithFailedTail } from "../../src/server"; import { relaySseEagerBounded, type EagerRelayHooks } from "../../src/server/relay-eager"; import { createForwardedEvidenceRedaction, createOutboundCredentialMask, createSynthesizedCopyMask, createTerminalErrorRedactionBlockRewrite } from "../../src/server/responses/terminal-error-redaction"; -import { OutboundCredentialRegistry, createSentValueGuard, maskSecretSpans, setOutboundCredentialRegistryForTests } from "../../src/lib/outbound-credential-registry"; +import { OutboundCredentialRegistry, createSentValueGuard, createSentValueMask, maskSecretSpans, setOutboundCredentialRegistryForTests } from "../../src/lib/outbound-credential-registry"; +import { REDACTED_SECRET } from "../../src/lib/redact"; +import { sseDataPayload } from "../../src/server/sse-payload-rewrite"; import { noteRelaySyntheticTail, noteRequestSentValueGuard } from "../../src/server/request-log-terminal-status"; import { reduceBareUpstreamEvidence } from "../../src/server/bare-upstream-error"; @@ -534,7 +536,8 @@ describe("untyped error envelopes never carry a value this request sent", () => test("excerpt masking of a large genuine terminal stays linear in the frame size", () => { // Every distinct matching window used to rescan the whole text: about 64k windows over a ~4 MB - // frame stalled the event loop for seconds. One marking pass keeps it to tens of milliseconds. + // frame stalled the event loop for seconds. Time that same excerpt masker directly; JSON/SSE + // serialization and the separate pattern masks are covered by the full rewrite checks below. const alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"; const credentials = [1, 2, 3, 4].map(seed => { let state = seed; @@ -542,14 +545,44 @@ describe("untyped error envelopes never carry a value this request sent", () => for (let i = 0; i < 16_000; i++) { state = (state * 1103515245 + 12345) & 0x7fffffff; value += alphabet[state % 62]; } return value; }); - const rewrite = createTerminalErrorRedactionBlockRewrite(Object.fromEntries(credentials.map((value, i) => [`x-api-key-${i}`, value]))); + const headers = Object.fromEntries(credentials.map((value, i) => [`x-api-key-${i}`, value])); + const rewrite = createTerminalErrorRedactionBlockRewrite(headers); + const mask = createSentValueMask(headers, REDACTED_SECRET, { credentialOnly: true, wholeText: true }); const echoed = credentials.map(value => value.slice(0, -1)).join(" "); - const failed = { type: "response.failed", response: { status: "failed", error: { code: "server_error", message: echoed + " " + "z".repeat(3_900_000 - echoed.length) } } }; + const suffix = "z".repeat(3_900_000 - echoed.length); + const message = echoed + " " + suffix; const startedAt = performance.now(); - const out = rewrite(`event: response.failed\ndata: ${JSON.stringify(failed)}`).join("\n"); + const masked = mask(message); const elapsed = performance.now() - startedAt; - expect(credentials.some(value => out.includes(value.slice(100, 140)))).toBe(false); + expect(credentials.some(value => masked.includes(value.slice(100, 140)))).toBe(false); + expect(masked.endsWith(" " + suffix)).toBe(true); expect(elapsed).toBeLessThan(1_500); + + const failed = { type: "response.failed", response: { status: "failed", error: { code: "server_error", message } } }; + const out = rewrite(`event: response.failed\ndata: ${JSON.stringify(failed)}`).join("\n"); + expect(credentials.some(value => out.includes(value.slice(100, 140)))).toBe(false); + const payload = sseDataPayload(out); + expect(payload).not.toBeNull(); + expect(JSON.parse(payload!).response.error.message).toBe(masked); + }); + + test("excerpt masking does not restart a whole-text search for every matching window", () => { + const credential = "abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ".repeat(4); + const suffix = "~".repeat(8192); + const text = credential.slice(0, -1) + " " + suffix; + const mask = createSentValueMask({ "x-api-key": credential }, REDACTED_SECRET, { credentialOnly: true, wholeText: true }); + const indexOf = String.prototype.indexOf; + let restarts = 0, masked = ""; + const scan = spyOn(String.prototype, "indexOf").mockImplementation(function (this: string, search: string, position?: number) { + if (String(this) === text && (position ?? 0) <= 0 && ++restarts > 8) { + throw new Error("excerpt masking repeatedly rescanned the whole input"); + } + return indexOf.call(this, search, position); + }); + try { masked = mask(text); } + finally { scan.mockRestore(); } + expect(masked).toBe(REDACTED_SECRET + " " + suffix); + expect(restarts).toBeLessThanOrEqual(8); }); test("exact masking of a self-overlapping credential stays linear in the frame size", () => { diff --git a/tests/server/server-auth.test.ts b/tests/server/server-auth.test.ts index 8aadb2aae3a..207bf55b7c8 100644 --- a/tests/server/server-auth.test.ts +++ b/tests/server/server-auth.test.ts @@ -60,7 +60,6 @@ import { getDebugLogEntries } from "../../src/lib/debug-log-buffer"; import { setDebugSettings } from "../../src/lib/debug-settings"; import { watchdogMs } from "../helpers/ci-watchdog"; import { removeTreeWithRetry } from "../helpers/remove-tree"; -import { deferredResetSseUpstream } from "../helpers/deferred-reset-sse-upstream"; import { serverAuthConfig as config } from "../helpers/server-auth-config"; import { POOL_RETRY_TEST_DIR, redirectCanonicalCodexTo } from "../helpers/pool-retry-harness"; const previousApiToken = process.env.OPENCODEX_API_AUTH_TOKEN; @@ -4093,14 +4092,19 @@ describe("server local API auth", () => { }, { timeout: SERVER_BUDGET_MS }); test("native passthrough upstream reset still logs 502 and penalizes the pool", async () => { - const upstream = deferredResetSseUpstream('data: {"type":"response.output_text.delta","delta":"hello"}\n\n'); - const harness = await startPoolRetryHarness(() => upstream.response(), { secondAccount: false, streamMode: "legacy-tee" }); + const harness = await startPoolRetryHarness(() => new Response(new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: {"type":"response.output_text.delta","delta":"hello"}\n\n')); + }, + }), { headers: { "content-type": "text/event-stream" } }), { secondAccount: false, streamMode: "legacy-tee" }); try { const response = await harness.request({ stream: true }); const requestId = response.headers.get("x-opencodex-request-id"); const reader = response.body!.getReader(); expect((await reader.read()).done).toBe(false); - upstream.reset(); + // Reset the owned upstream socket after its first chunk. Throwing from a + // fixture stream instead makes Bun's server sink report an uncaught error. + await harness.upstream.stop(true); while (!(await reader.read()).done) { /* drain the synthetic failed terminal */ } const deadline = Date.now() + INTERNAL_DEADLINE_MS; while (!getRequestLogEntries().some(entry => entry.requestId === requestId) && Date.now() < deadline) {