Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -627,6 +627,9 @@ jobs:
cd gui
bun install --frozen-lockfile

- name: Gateway quota and cache guardrails
run: bun run test:gateway-quota

- name: GUI lint
if: needs.changes.outputs.gui == 'true'
run: |
Expand Down
12 changes: 12 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,18 @@ Source development requires the `bun` CLI on your `PATH`. The published npm pack
Bun runtime for end users, but contributor commands such as `bun install`, `bun run test`, and
`bun run prepush` run from your local Bun installation.

## AgentRadar fork quota and cache checks

On the hosted fork release line, run `bun run test:gateway-quota` before proposing gateway changes.
It runs the existing offline account-pin, quota-refusal, physical-send, replay-prefix, usage and
cache-stabilization suites together through the isolated test runner. Provider traffic is synthetic;
these checks do not measure a live cache-hit rate or spend a subscription.

The `Gateway quota and cache guardrails` step is part of the existing Cross-platform CI `gates`
job, so a failing suite fails the aggregate `ci` check. Fork publication already requires successful
push-event Cross-platform CI for the exact release SHA. This does not replace the broader tests,
review or the repository's branch rules.

## Pull request contract

A ready-for-review PR is the author's claim that the change is complete, understood, tested, and suitable for merging. Opening a PR does not transfer responsibility for the branch to maintainers.
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
"start": "bun run src/cli/index.ts start",
"test": "bun scripts/test.ts",
"test:changed": "bun scripts/test.ts --changed=dev",
"test:gateway-quota": "bun scripts/test.ts tests/adapters/anthropic/anthropic-quota-dispatch.test.ts tests/adapters/anthropic/anthropic-pool-extra-usage-400.test.ts tests/adapters/anthropic/anthropic-account-pool.test.ts tests/adapters/anthropic/anthropic-replay-oauth-identity.test.ts tests/adapters/anthropic/anthropic-replay-combo.test.ts tests/server/agentradar-account-pin-e2e.test.ts tests/responses/responses-send-budget-counts.test.ts tests/adapters/adapter-inner-send-budget.test.ts tests/usage/key-attribution.test.ts tests/claude-integration/claude-inbound-cache-stabilize.test.ts tests/server/terminal-guard-server.test.ts",
"typecheck": "bun x tsc --noEmit",
"audit:high": "bun audit --audit-level=high && cd gui && bun audit --audit-level=high",
"privacy:scan": "bun scripts/privacy-scan.ts",
Expand Down
7 changes: 5 additions & 2 deletions src/server/responses/adapter-continuation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ export function createAdapterContinuations(
| "applyFailoverSnapshot"
| "replayOAuthCredentialSnapshot"
| "noteRoutedAttemptSend"
| "anthropicPhysicalBudget"
| "anthropicSendAvailable"
>,
sidecarState: Pick<ResponsesSidecarAuth, "routedCompaction">,
sendBudgetState: Pick<
Expand Down Expand Up @@ -257,7 +259,7 @@ export function createAdapterContinuations(
...(continuationTransientPolicy
? {
attempts: remainingTransientSendBudget(continuationTransientPolicy.attempts),
onSendsConsumed: noteTransientSends,
onSendsConsumed: transportState.anthropicPhysicalBudget ? undefined : noteTransientSends,
}
: {}),
},
Expand All @@ -283,7 +285,7 @@ export function createAdapterContinuations(
// A credential denial (403 oauth_org_not_allowed) rotates the Anthropic account here too:
// the continuation is a fresh send, so a denied credential is recoverable rather than
// terminal. `nextParsed` is the terminal-guard clone; both owners are rebound.
const deniedCredentialRecovery = await rotateAnthropicProviderOnCredentialDenial({
const deniedCredentialRecovery = !transportState.anthropicSendAvailable() ? null : await rotateAnthropicProviderOnCredentialDenial({
config, route, inboundWire, logCtx, transportState, parsed,
retryParsed: nextParsed,
response,
Expand Down Expand Up @@ -391,6 +393,7 @@ export function createAdapterContinuations(
transportState.anthropicPoolAccountId
&& !isNonReplayableResponse(response)
&& transportState.anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST
&& transportState.anthropicSendAvailable()
) {
try {
const peeked = await peekAnthropicPoolRotation(response, upstream.signal);
Expand Down
24 changes: 14 additions & 10 deletions src/server/responses/adapter-dispatch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,9 @@ export async function prepareAdapterExchange(
| "genericFailovers"
| "applyFailoverSnapshot"
| "noteRoutedAttemptSend"
| "anthropicPhysicalBudget"
| "anthropicSendAvailable"
| "setAnthropicDispatchBudget"
>,
responseEffects: Pick<ResponsesEffects, "cancelResponseCompletion" | "notifyResponseComplete" | "refreshRequestToolAliases">,
sendBudgetState: Pick<
Expand Down Expand Up @@ -178,7 +181,7 @@ export async function prepareAdapterExchange(
sendBudgetExhausted,
reserveCredentialHop,
} = sendBudgetState;

transportState.setAnthropicDispatchBudget(adapterDispatchBudget);

const upstream = new AbortController();
const cleanupUpstreamAbort = linkAbortSignal(upstream, options.abortSignal);
Expand Down Expand Up @@ -359,7 +362,7 @@ export async function prepareAdapterExchange(
// shared counter was inherited without ever being read as a limit.
? {
attempts: remainingTransientSendBudget(transientPolicy.attempts),
onSendsConsumed: noteTransientSends,
onSendsConsumed: transportState.anthropicPhysicalBudget ? undefined : noteTransientSends,
}
: {}),
},
Expand Down Expand Up @@ -455,7 +458,7 @@ export async function prepareAdapterExchange(
// their physical-send boundary, without booking the request ledger again.
let fastDowngradeWorkflowCharged = false;
const chargeFastDowngradeWorkflowSend = (): void => {
if (recovery !== "anthropic-fast-downgrade" || fastDowngradeWorkflowCharged) return;
if (transportState.anthropicPhysicalBudget || recovery !== "anthropic-fast-downgrade" || fastDowngradeWorkflowCharged) return;
fastDowngradeWorkflowCharged = true;
chargeWorkflowSends(sendBudgetState.workflowRootId, 1);
};
Expand Down Expand Up @@ -537,7 +540,7 @@ export async function prepareAdapterExchange(
// Same rule as the passthrough rebuild: spend the base allowance first, then the one
// shared final-recovery reserve, so a recovery that follows a spent streak still gets
// its single send instead of dying at three.
const refetchAllowance = refetchTransientPolicy
const refetchAllowance = refetchTransientPolicy && !transportState.anthropicPhysicalBudget
? recoverySendAllowance(
refetchTransientPolicy.attempts,
recoveryClassFor(recovery),
Expand Down Expand Up @@ -571,7 +574,7 @@ export async function prepareAdapterExchange(
...(refetchAllowance
? {
attempts: refetchAllowance.attempts,
onSendsConsumed: noteTransientSends,
onSendsConsumed: transportState.anthropicPhysicalBudget ? undefined : noteTransientSends,
}
: {}),
},
Expand Down Expand Up @@ -704,7 +707,7 @@ export async function prepareAdapterExchange(

// An Anthropic OAuth credential denial (403 oauth_org_not_allowed) is not a quota verdict:
// the account is marked for reauth and the turn continues on the next eligible account.
const deniedCredentialRecovery = await rotateAnthropicProviderOnCredentialDenial({
const deniedCredentialRecovery = !transportState.anthropicSendAvailable() ? null : await rotateAnthropicProviderOnCredentialDenial({
config, route, inboundWire, logCtx, transportState, parsed,
response: upstreamResponse,
signal: upstream.signal,
Expand Down Expand Up @@ -735,8 +738,8 @@ export async function prepareAdapterExchange(
const adapterOwnsDispatch = transportState.activeAdapter.fetchResponse !== undefined;
const hop = reserveCredentialHop(
"repair",
`${route.providerName}|${route.modelId}|anthropic-fast-downgrade`,
!adapterOwnsDispatch && transientRetryPolicyFor(route.provider) !== null,
`${route.providerName}|${route.modelId}|${transportState.anthropicPhysicalBudget ? "anthropic-physical" : "anthropic-fast-downgrade"}`,
!transportState.anthropicPhysicalBudget && !adapterOwnsDispatch && transientRetryPolicyFor(route.provider) !== null,
);
if (hop.allowed) {
anthropicFastDowngradeGuard.attempted = true;
Expand All @@ -751,7 +754,7 @@ export async function prepareAdapterExchange(
let result: Response | { failed: Response };
try {
result = await rebuildAndRefetch("anthropic-fast-downgrade", () => {
if (!adapterOwnsDispatch) hop.permit?.use();
if (!adapterOwnsDispatch && !transportState.anthropicPhysicalBudget) hop.permit?.use();
});
} finally {
sendBudgetState.pendingHopPermit = undefined;
Expand Down Expand Up @@ -869,6 +872,7 @@ export async function prepareAdapterExchange(
while (
transportState.anthropicPoolAccountId
&& transportState.anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST
&& transportState.anthropicSendAvailable()
) {
let poolRotation: Awaited<ReturnType<typeof peekAnthropicPoolRotation>>;
try {
Expand Down Expand Up @@ -1012,7 +1016,7 @@ export async function prepareAdapterExchange(
// and settling here first would hand it a dead permit, which it reads as an
// exhausted request and stops sending on.
result = await rebuildAndRefetch("oauth-account-429", () => {
if (!adapterOwnsDispatch) hop.permit?.use();
if (!adapterOwnsDispatch && !transportState.anthropicPhysicalBudget) hop.permit?.use();
});
} finally {
sendBudgetState.pendingHopPermit = undefined;
Expand Down
33 changes: 30 additions & 3 deletions src/server/responses/request-transport.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options";
import { CODEX_TEXT_GUARDED_BUDGET_POLICY, deriveRequestExecutionBudget, isRequestExecutionBudget, type RequestExecutionBudget } from "../../lib/request-execution-budget";
import { SendBudgetExhaustedError } from "../../lib/upstream-retry";
import { chargeWorkflowSends } from "../../lib/workflow-budget";
import type { PreparedResponsesRequest } from "./request-prepare";
import type { OAuthAccessSnapshot } from "../../oauth";
import {
Expand All @@ -11,6 +14,7 @@ import type { ProviderAdapter, AdapterRequest } from "../../adapters/base";
import type { AdapterEvent, OcxParsedRequest, OcxProviderConfig, OcxUsage } from "../../types";
import type { AnthropicAccountSelectionReason } from "../../oauth/anthropic-routing";
import {
ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST,
isAnthropicAccountPoolEnabled,
getAnthropicPoolAccessSnapshot,
commitAnthropicSelectionRouting,
Expand Down Expand Up @@ -90,6 +94,18 @@ export async function prepareResponsesTransport(
) {
const { config, logCtx, options, req } = requestContext;
const { route, parsed, inboundWire, translatorBudget } = requestState;
const anthropicPhysicalBudget = route.providerName === "anthropic" && route.provider.authMode === "oauth";
if (anthropicPhysicalBudget && isRequestExecutionBudget(options.sendBudget)
&& options.sendBudget.policy === CODEX_TEXT_GUARDED_BUDGET_POLICY) {
// Keep the fork's existing 16-account recovery allowance; never widen an explicit policy.
options.sendBudget = deriveRequestExecutionBudget(options.sendBudget, {
...options.sendBudget.policy, maxTotalModelSends: ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST + 1,
baseSendAllowance: ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST + 1,
});
}
let anthropicDispatchBudget = isRequestExecutionBudget(options.sendBudget) ? options.sendBudget : undefined;
const anthropicSendAvailable = (): boolean => !anthropicPhysicalBudget || !anthropicDispatchBudget
|| anthropicDispatchBudget.used < anthropicDispatchBudget.policy.maxTotalModelSends;
// AgentRadar image patch: the pinned account's own snapshot and credential generation are the
// only admission; the global selection, affinity and pool cursors are never read or written.
const pin = route.provider.authMode === "oauth" ? options.agentRadarAccount : undefined;
Expand Down Expand Up @@ -275,11 +291,11 @@ export async function prepareResponsesTransport(
// physical dispatch binds it to the selection that actually reaches the upstream.
let pendingKeySend: { estimate: number | undefined; recovery?: AttemptRecoveryKind } | undefined;
const noteRoutedAttemptSend = (estimate: number | undefined, recovery?: AttemptRecoveryKind): void => {
if (usesApiKeyAccount(route.provider)) pendingKeySend = { estimate, recovery };
if (anthropicPhysicalBudget || usesApiKeyAccount(route.provider)) pendingKeySend = { estimate, recovery };
else noteProviderAttemptSend(logCtx, route.providerName, route.provider, estimate, recovery);
};
const commitKeyAttemptSend = (): void => {
if (!usesApiKeyAccount(route.provider)) return;
if (!anthropicPhysicalBudget && !usesApiKeyAccount(route.provider)) return;
noteProviderAttemptSend(logCtx, route.providerName, route.provider,
pendingKeySend?.estimate ?? logCtx.usageLogInputTokens, pendingKeySend?.recovery);
pendingKeySend = undefined;
Expand Down Expand Up @@ -453,11 +469,19 @@ export async function prepareResponsesTransport(
// Reselection can choose a provider override instead of the supplied executor.
// Either way the send crosses the physical boundary, so the connection policy is
// applied around whichever implementation was just selected (#4992).
commitKeyAttemptSend();
// The binding travels with the send, so a rebuilt request resolves its provider route
// against the destination it is actually going to rather than the one this dispatch
// started with. Account reselection can move the upstream host, which would otherwise
// apply a host-scoped decision to a different host.
if (anthropicPhysicalBudget && anthropicDispatchBudget) {
const dispatch = anthropicDispatchBudget.reserveDispatch({
sendClass: anthropicDispatchBudget.used === 0 ? "initial" : "auth-recovery",
targetKey: `${route.providerName}|${route.modelId}|anthropic-physical`,
});
if (!dispatch.allowed || !dispatch.permit.use()) throw new SendBudgetExhaustedError("Anthropic");
chargeWorkflowSends(req.headers.get("x-codex-parent-thread-id")?.trim() || undefined, 1);
}
commitKeyAttemptSend();
const response = await sendWithConnectionPolicy(
fetchImpl,
destination,
Expand Down Expand Up @@ -912,6 +936,9 @@ export async function prepareResponsesTransport(
anthropicMessagesSource = undefined;
},
oauthDispatch,
anthropicPhysicalBudget,
anthropicSendAvailable,
setAnthropicDispatchBudget(budget: RequestExecutionBudget | undefined): void { anthropicDispatchBudget = budget; },
noteRoutedAttemptSend,
commitKeyAttemptSend,
bindKeyUsageFromBridge,
Expand Down
13 changes: 8 additions & 5 deletions src/server/responses/sidecar-execution.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ export async function executeResponsesSidecars(
| "noteRoutedAttemptSend"
| "bindKeyUsageFromBridge"
| "exactAnthropicSourceReplay"
| "anthropicPhysicalBudget"
| "anthropicSendAvailable"
| "setAnthropicDispatchBudget"
>,
sidecarState: Pick<ResponsesSidecarAuth, "routedCompaction" | "openAiSidecar">,
responseEffects: Pick<
Expand All @@ -81,7 +84,7 @@ export async function executeResponsesSidecars(
| "notifyResponseComplete"
| "cancelResponseCompletion"
>,
sendBudgetState: Pick<ResponsesSendBudget, "reserveCredentialHop">,
sendBudgetState: Pick<ResponsesSendBudget, "reserveCredentialHop" | "adapterDispatchBudget">,
) {
const { config, options, logCtx } = requestContext;
const {
Expand All @@ -101,6 +104,7 @@ export async function executeResponsesSidecars(
} = requestState;
const { routedCompaction, openAiSidecar } = sidecarState;
const { reserveCredentialHop } = sendBudgetState;
transportState.setAnthropicDispatchBudget(sendBudgetState.adapterDispatchBudget);
const {
commitReasoningReplayServingRoute,
continuationStateForResponse,
Expand Down Expand Up @@ -237,10 +241,9 @@ export async function executeResponsesSidecars(
) {
// Same intersection for the Anthropic roster: its own per-request bound still applies,
// and the shared budget decides whether this request may spend another send at all.
const hop = reserveCredentialHop(
"auth-recovery",
`${route.providerName}|${route.modelId}|sidecar-anthropic-429`,
);
const hop = transportState.anthropicPhysicalBudget
? { allowed: transportState.anthropicSendAvailable(), permit: undefined }
: reserveCredentialHop("auth-recovery", `${route.providerName}|${route.modelId}|sidecar-anthropic-429`);
if (!hop.allowed) return null;
const nextAccountId = rotateAnthropicAccountOn429(
config,
Expand Down
4 changes: 3 additions & 1 deletion structure/transports/responses-failover.md
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,9 @@ In `adapter-dispatch.ts` the guard at the top of the recovery loop is necessary
sufficient. The refusal can also be produced by a refetch made INSIDE an arm, and that arm
then still holds it: the same-target loop re-enters while `rateLimitRetries` is below the
configured attempts, and the key, Anthropic-pool and generic-OAuth rotations re-enter while a
credential is left to try. The key-401 arm is in the same class from the other direction — its
credential is left to try. Hosted Anthropic rotations also follow the
[shared physical-send contract](responses-spend.md#hosted-anthropic-physical-sends): no funded
replacement means the current refusal remains authoritative. The key-401 arm is in the same class from the other direction — its
refetch answers 429 and it falls through into the arms below. So every arm that reassigns
`upstreamResponse` from `rebuildAndRefetch` re-enters the loop guard rather than continuing,
which is what makes the top-of-loop check the single exit for this verdict.
Expand Down
14 changes: 14 additions & 0 deletions structure/transports/responses-spend.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,20 @@ derivation agree rather than each settling against a counter the other cannot se
What stays per-scope is deliberate: the reserve, alternate-target and transition ledgers are each
target's own recovery decision, while the physical-send total is what binds every target together.

## Hosted Anthropic physical sends

The fork charges Anthropic OAuth inference at the `oauthDispatch` physical boundary in
`src/server/responses/request-transport.ts`, including initial, continuation and sidecar sends.
The boundary uses the shared adapter budget view, so an existing hop reservation is consumed
rather than charged twice; helper reporters do not charge that same send again. A zero allowance
prevents inference, and an unfunded account replacement retains the preceding upstream refusal.
Only the factory's unchanged default guarded policy is expanded for the fork's existing account
walk. Explicit policies and inherited custom ceilings are never widened or reset.
`tests/adapters/anthropic/anthropic-quota-dispatch.test.ts` counts mock-provider calls for zero,
one-send, funded replacement, inherited-budget, continuation and sidecar cases. The fork's
`bun run test:gateway-quota` also exercises pin isolation, replay cache-prefix stability and usage
accounting in the existing CI `gates` job; it makes no live cache-hit or paid-provider claim.

## Durable spend reservations

The request's send budget bounds how many times it may reach upstream; the spend ledger bounds
Expand Down
Loading