Repository navigation
feat(providers): park a key until the reset instant the upstream declared - #4733
Closed
abhisheksharma2411 wants to merge 3 commits into
Closed
abhisheksharma2411 wants to merge 3 commits into
abhisheksharma2411 wants to merge 3 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4024.
Summary
A free-tier quota exhaustion is dated — OpenRouter replies
Weekly/Monthly Limit Exhausted ... will reset at <date>. But it puts that date in the body, and sends noRetry-After.parseRetryAfterMsonly reads the header, so the key falls back toDEFAULT_COOLDOWN_MS, returns after a minute, takes another 429, rotates again — every minute, for the rest of the quota window. @Ingwannu confirmed the same reading ofkey-failover.tsondev@29bb221c3.parseQuotaResetAt(body)— pullsreset[s] at <date>from a bounded 4 KB prefix.readQuotaResetAt(response)— reads it off a cloned response, so the caller can stillbody.cancel()to release the socket, whichadapter-dispatch.tsdoes on the next line.The cap, and why it is not the existing one
MAX_COOLDOWN_MSstays 10 minutes. That shortness is correct for an undated 429 — it's a guess, and a guess shouldn't park a working key for long. A dated one isn't a guess, so it gets its own ceiling:MAX_QUOTA_COOLDOWN_MS = 32 days.I had that wrong at first and the test caught it. I wrote 8 days, reasoning about the weekly case in the issue. But the wording being parsed is
Weekly/**Monthly** Limit Exhausted, and an 8-day cap silently clamps every monthly reset ~23 days early — straight back into the loop this exists to remove. 31 days plus a day of slack for month length and zone.Bounded at all because the date is upstream-controlled input:
reset at 2999-01-01must not park a working key past any horizon an operator would think to look at.A runtime divergence worth knowing about
The parser pins a bare
YYYY-MM-DD hh:mm:ssto UTC explicitly. That looks redundant on this runtime, and I want to be straight about why it isn't:ECMA-262 says a date-time form with no offset is local; Node follows it, Bun currently doesn't. So on Bun the normalisation is a no-op today — and if Bun ever conforms, an un-normalised parse would shift every park-until by the operator's offset, with the early direction resuming the 429 tax.
The honest consequence: no Bun test can observe that branch being removed. I mutated it away and the suite stayed green. The explicit-zone case (
+05:30) is the part the suite can pin, and it does. Flagging rather than claiming coverage I don't have.Verification
Typecheck failures are
src/server/responses/fetch-helpers.ts(195,7)and(208,7),'timeout' does not exist in type 'RequestInit'— same two as on untoucheddev.Mutation-tested:
Scope
Wired into
adapter-dispatch.ts, the main rotation path. The otherrotateProviderTransportOn429call sites (chat-native,adapter-continuation,encrypted-payload,compact) pass nothing and keep exactly today's behaviour — the option is optional, so this is additive. Happy to thread the rest in this PR or a follow-up; I stopped at one path rather than touch five hot sites in a change that needs live-traffic confirmation anyway.The issue's second half — reset-aware combo target parking, as opposed to key parking — is not here. That's a different mechanism and @Ingwannu's note about rotation persisting the replacement key suggests it wants its own discussion.
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
All CI tests are green on my local testing.
I pushed my PR to the latest dev commit.
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.
Summary by CodeRabbit