Skip to content

fix(anthropic): hop past a spent subscription's "out of extra usage" 400 - #8649

Closed
andrew05060414 wants to merge 1 commit into
lidge-jun:devfrom
andrew05060414:fix/anthropic-extra-usage-hop
Closed

andrew05060414 wants to merge 1 commit into
lidge-jun:devfrom
andrew05060414:fix/anthropic-extra-usage-hop

Conversation

@andrew05060414

@andrew05060414 andrew05060414 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A Claude subscription whose included usage and extra-usage balance are both spent answers HTTP 400 invalid_request_error with the exact sentence:

You're out of extra usage. Add more at claude.ai/settings/usage and keep going.

Today the combo reads that 400 as a malformed request and stops, so the targets behind the exhausted subscription (an anthropic API-key lane, another vendor) are never tried and the client sees the raw upstream error. For anyone running a "subscription first, then pay-as-you-go" combo, the failover list stops working the moment the subscription runs dry.

This PR treats that whole-message sentence as billing evidence, the same way isAnthropicBillingRefusal already treats the credit-balance sentence: the combo hops to the next target and cools the provider. Both apostrophe forms (' and ’) are accepted, the trailing "Add more at …" clause is optional, and the existing guards still apply. Echoed or decorated prose (a validation 400 that merely quotes the sentence, a prefixed diagnostic, an appended clause) is not evidence, so those keep stopping the chain, and a 429 carrying the text is unaffected.

Files:

  • src/providers/anthropic-billing-refusal.ts: one more whole-message pattern.
  • tests/adapters/anthropic/anthropic-billing-refusal.test.ts: accepted and rejected shapes.
  • tests/codex-integration/combo-anthropic-extra-usage.test.ts (new): comboFailureDecision / comboFailureCooldownScope give hop / provider for the refusal and stop / target for a quoting validation error. Registered in scripts/test-layout/layout-combos.json and tests/fixtures/test-layout-expected.json.

No docs change: the user-facing contract ("a billing refusal advances the combo") is unchanged; this widens which upstream sentence counts as one.

Verification

  • bun run typecheck: exit 0 (no type errors in 2,492 files).
  • bun test tests/adapters/anthropic/anthropic-billing-refusal.test.ts tests/codex-integration/combo-anthropic-extra-usage.test.ts tests/test-layout.test.ts: all pass.
  • bun test tests/test-layout-tooling.test.ts --timeout 60000: 27 pass. With the default 5 s timeout, move end to end times out on this Windows machine, independent of this change (it runs git in a temp repo).
  • bun run structure:check: exit 0. bun run privacy:scan: exit 0.
  • bun run test:changed: not a clean result. The import graph of anthropic-billing-refusal.ts reaches a large part of the suite, and the run hit the runner's time limit (exit 124) on a Windows machine shared with other worktrees' test runs. It reported 47 failures, 43 of them this test timed out after 5000ms in suites unrelated to this change (link WebSocket, pool retry, client config convergence, etc.). The full suite was not run locally for the same contention reason; that coverage is left to CI.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

A Claude subscription whose included usage and extra-usage balance are both
spent answers HTTP 400 invalid_request_error with the exact sentence
"You're out of extra usage. Add more at claude.ai/settings/usage and keep
going." The combo read that 400 as a malformed request and stopped, so the
targets behind the exhausted account (an API-key lane, another vendor) were
never tried and the client saw the raw upstream error.

Treat the whole-message sentence as billing evidence, the same way the
credit-balance sentence already is: the combo hops and cools the provider.
Echoed or decorated prose still is not authority, so a validation 400 that
quotes the sentence keeps stopping the chain.
@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Nov 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

@Ingwannu Ingwannu left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested for d5e0b745dced1690ba7f1b1f4c2eee63947e5b8f.

The five-file diff extends the shared isAnthropicBillingRefusal predicate beyond credit-balance refusals but changes no owning structure documentation. Record the new extra-usage evidence and its consequences in the billing/Combo contract. In particular, this branch treats the refusal as provider-scoped (the new regression asserts "provider"), whereas open #8588 treats overlapping exact wording as target-scoped and additionally handles the third-party-apps sentence. These are different cooldown/account-health decisions, not interchangeable duplicate patches. Please coordinate the intended evidence grammar and cooldown scope before landing both; do not close either on title similarity alone.

Also make the intended envelope scope explicit in tests. Returning the new regex result from the existing shared predicate inherits its accepted statuses/types/codes (400/402, billing_error/permission_error and account-refusal codes), while the documented observed case here is HTTP400 invalid_request_error. Preserve the old credit-balance behavior and either narrow the new message to its justified carrier or document/test evidence for broader carriers. Include negative controls for quoted/decorated text and conflicting codes; #8588's narrower independent helper is one reference, not a required implementation.

I allowed only the existing matching-head hosted CI after reviewing the full diff and execution surfaces. That permission does not choose the policy, grant sponsorship or approve a merge. Local execution was deferred because other Cargo workloads occupy the shared host; no live subscription probe or scan was run.

@Ingwannu Ingwannu left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI follow-up for d5e0b745dced1690ba7f1b1f4c2eee63947e5b8f: the approved Cross-platform run 43368835353 completed red. The actual failed job is test 5/5, batch 5/62: tests/ci-workflows/docs-spanish-localization.test.ts:277, “keeps executable examples and frontmatter in the current English shape”. That batch reports 320 pass / 1 fail; the executable-fence comparison differs around the update-command failure/Windows Get-Process example. The other four Ubuntu shards, gates and Catalog check passed. Native Windows/macOS jobs were not selected.

This log does not show a failure in your new Anthropic refusal case, and does not by itself prove the docs failure is unrelated or already fixed. Please account for it with a focused current-head comparison or the linked correction before readiness; preserve the executable-example contract rather than skipping the assertion. I have not rerun the run. My earlier change request about accepted carriers, the owning documentation, and the target-vs-provider cooldown overlap with #8588 remains open.

@andrew05060414

Copy link
Copy Markdown
Contributor Author

Closing this in favour of #8588, after comparing the two against the review above.

Why #8588 and not this one

  • fix(combos): hop past Anthropic's extra-usage refusal #8588 already makes the decisions the review asked this PR to make: it keeps the extra-usage evidence in its own narrow helper (isAnthropicExtraUsageFailure), accepts only HTTP 400 invalid_request_error with no account-refusal code, and leaves the old credit-balance predicate untouched.
  • It also covers the second wording ("Third-party apps now draw from your extra usage, not your plan limits."), which this PR does not.
  • It records the contract in structure/runtime.md and has negative controls for quoted, coded, other-typed and uncoded bare-prose variants.
  • On cooldown scope I agree with fix(combos): hop past Anthropic's extra-usage refusal #8588's reasoning: the third-party wording can depend on the request shape, so a target-scoped hop is the safer default. This PR's provider-scoped assertion was the wrong call.

Checked locally

I ran this PR's regression case (You're out of extra usage. Add more at claude.ai/settings/usage and keep going. in an invalid_request_error envelope, codes undefined / null / invalid_request_error, plus the quoted-sentence negative control) against #8588's head 7460c82ca, with the expected scope changed from provider to target. The decisions match: hop for the exact envelope, stop for the quoted sentence (6 pass). The one reported failure in that run is not an assertion: on this Windows host the preload's .git/config guard cannot run git config with GIT_CONFIG_GLOBAL=NUL (git 2.56 answers fatal: unable to access 'NUL'), so the first case of every file is charged with the guard's arm-time error.

Nothing in this PR is needed on top of #8588, so I am closing it rather than asking for a second review of overlapping code. The test 5/5 failure noted above was in docs-spanish-localization and unrelated to this diff.

andrew05060414 added a commit to andrew05060414/opencodex that referenced this pull request Nov 13, 2026
…usage hop

lidge-jun#8649 was closed in favour of lidge-jun#8588, which keeps the evidence in its own narrow helper, covers the third-party-apps wording too, and keeps the cooldown target-scoped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants