Skip to content

feat(providers): park a key until the reset instant the upstream declared - #4733

Closed
abhisheksharma2411 wants to merge 3 commits into
lidge-jun:devfrom
abhisheksharma2411:feat/openrouter-quota-reset-cooldown
Closed

abhisheksharma2411 wants to merge 3 commits into
lidge-jun:devfrom
abhisheksharma2411:feat/openrouter-quota-reset-cooldown

Conversation

@abhisheksharma2411

@abhisheksharma2411 abhisheksharma2411 commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Closes #4024.

Summary

A free-tier quota exhaustion is dated — OpenRouter replies Weekly/Monthly Limit Exhausted ... will reset at <date>. But it puts that date in the body, and sends no Retry-After. parseRetryAfterMs only reads the header, so the key falls back to DEFAULT_COOLDOWN_MS, returns after a minute, takes another 429, rotates again — every minute, for the rest of the quota window. @Ingwannu confirmed the same reading of key-failover.ts on dev@29bb221c3.

  • parseQuotaResetAt(body) — pulls reset[s] at <date> from a bounded 4 KB prefix.
  • readQuotaResetAt(response) — reads it off a cloned response, so the caller can still body.cancel() to release the socket, which adapter-dispatch.ts does on the next line.
  • When present, it outranks both the header and the default. It is the only one of the three that knows when the quota actually comes back.

The cap, and why it is not the existing one

MAX_COOLDOWN_MS stays 10 minutes. That shortness is correct for an undated 429 — it's a guess, and a guess shouldn't park a working key for long. A dated one isn't a guess, so it gets its own ceiling: MAX_QUOTA_COOLDOWN_MS = 32 days.

I had that wrong at first and the test caught it. I wrote 8 days, reasoning about the weekly case in the issue. But the wording being parsed is Weekly/**Monthly** Limit Exhausted, and an 8-day cap silently clamps every monthly reset ~23 days early — straight back into the loop this exists to remove. 31 days plus a day of slack for month length and zone.

Bounded at all because the date is upstream-controlled input: reset at 2999-01-01 must not park a working key past any horizon an operator would think to look at.

A runtime divergence worth knowing about

The parser pins a bare YYYY-MM-DD hh:mm:ss to UTC explicitly. That looks redundant on this runtime, and I want to be straight about why it isn't:

Node  Date.parse("2026-09-09 03:30:06")  →  2026-09-09T10:30:06Z   (local, PDT box)
Bun   Date.parse("2026-09-09 03:30:06")  →  2026-09-09T03:30:06Z   (UTC)

ECMA-262 says a date-time form with no offset is local; Node follows it, Bun currently doesn't. So on Bun the normalisation is a no-op today — and if Bun ever conforms, an un-normalised parse would shift every park-until by the operator's offset, with the early direction resuming the 429 tax.

The honest consequence: no Bun test can observe that branch being removed. I mutated it away and the suite stayed green. The explicit-zone case (+05:30) is the part the suite can pin, and it does. Flagging rather than claiming coverage I don't have.

Verification

openrouter-quota-reset-cooldown-4024.test.ts                         10 pass, 0 fail
every other suite importing key-failover (5 files)                  105 pass, 0 fail
bun run structure:check                                             passed
bun run privacy:scan                                                passed
bun run typecheck                                                   2 errors — PRE-EXISTING

Typecheck failures are src/server/responses/fetch-helpers.ts(195,7) and (208,7), 'timeout' does not exist in type 'RequestInit' — same two as on untouched dev.

Mutation-tested:

Mutation Result
ignore the body date entirely (today's behaviour) 6 fail
remove the 32-day cap cap test fails
scan the whole body instead of 4 KB prefix test fails
drop the UTC normalisation survives — see above
restored 10 pass

Scope

Wired into adapter-dispatch.ts, the main rotation path. The other rotateProviderTransportOn429 call sites (chat-native, adapter-continuation, encrypted-payload, compact) pass nothing and keep exactly today's behaviour — the option is optional, so this is additive. Happy to thread the rest in this PR or a follow-up; I stopped at one path rather than touch five hot sites in a change that needs live-traffic confirmation anyway.

The issue's second half — reset-aware combo target parking, as opposed to key parking — is not here. That's a different mechanism and @Ingwannu's note about rotation persisting the replacement key suggests it wants its own discussion.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • All CI tests are green on my local testing.

  • I pushed my PR to the latest dev commit.

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Summary by CodeRabbit

  • Bug Fixes
    • Improved provider failover when quota limits are reported in a response body.
    • Quota reset times are now recognized and used to schedule key cooldowns, including UTC, date-only, and time-zone-aware timestamps.
    • Reset schedules take precedence over generic retry timing, with safeguards for invalid, expired, or excessively distant dates.
    • Response bodies remain available after reset-time detection.
    • Existing behavior remains unchanged when reset details are unavailable or for authentication failures.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request landed-via-maintainer Original PR closed after landing via a maintainer merge train

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants