Skip to content

fix(google): release a superseded retry response before waiting for a pacing slot - #8743

Merged
lidge-jun merged 1 commit into
lidge-jun:devfrom
Caledvwlch:codex/google-retry-release-lease-before-pacing
Nov 19, 2026
Merged

lidge-jun merged 1 commit into
lidge-jun:devfrom
Caledvwlch:codex/google-retry-release-lease-before-pacing

Conversation

@Caledvwlch

@Caledvwlch Caledvwlch commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

With requestPacing.maxConcurrentRequests on a Google provider, a retried request could wait on itself. When a send failed with a retryable status, fetchGoogleWithRetry kept the previous response open and then asked the pacer for a new slot. The backoff and the cancellation of the old response both ran in beforeDispatch, which only runs after the slot is granted. With the cap saturated, the slot the retry was waiting for was held by its own previous response, so the turn stalled until a deadline fired. When a slot was free, the retry held the new slot through the whole backoff and blocked other requests.

Example: maxConcurrentRequests: 1, the first send returns 503. Before this change, the retry queues behind the slot owned by that 503 response and never dispatches. After it, the 503 body is cancelled (releasing its slot), the backoff runs holding no slot, and the retry then takes the free slot and dispatches.

Changes:

  • src/adapters/physical-send.ts: createAdapterPhysicalSend gains an optional beforePacing callback. It runs after the dispatch budget is reserved and before waitForPacing, followed by an abort check. beforeDispatch keeps its post-admission meaning.
  • src/adapters/google-http.ts: the retry preparation moves to beforePacing. It cancels the previous response body first, then runs the abort-aware backoff. On a send-budget refusal the retained response is handed to the caller (Retry-After stays available) and the loop gives up ownership. A finally cancels a response the loop still owns when the request is cancelled before the next send.
  • Docs: structure/adapters/registry.md (new "Google retry lease ordering" section) and a pointer in structure/providers-and-adapters.md.

No account, model, deadline, retry budget or concurrency limit changes.

Verification

New tests: tests/adapters/google/google-retry-before-pacing.test.ts, tests/adapters/google/google-retry-held-lease.test.ts (saturated five-response case), and additions to tests/adapters/physical-send.test.ts.

  • On dev (873ec9b): 7 failures, 4 of them 5-second timeouts showing the self-wait.
  • With this change: 39 pass, 0 fail.
bun scripts/test.ts --timeout 30000 tests/adapters/google/google-retry-before-pacing.test.ts \
  tests/adapters/google/google-retry-held-lease.test.ts tests/adapters/physical-send.test.ts

Wider check through the repo runner (bun scripts/test.ts, isolated home and --isolate) on all of tests/adapters/google plus tests/adapters/physical-send.test.ts: 1088 pass / 0 fail with this change, 1064 pass / 0 fail on dev. The 24 extra passes are the new tests (17 + 3 in the new files, 4 added to physical-send.test.ts). The full suite was not run locally.

Also passing: bun check --no-script --noEmit (0 errors), scripts/structure-ssot.ts, scripts/privacy-scan.ts.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (No auth, workflow or dependency surface is touched.)

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Summary by CodeRabbit

  • Bug Fixes
    • Google retries now release the previous response before waiting for pacing or retry backoff, helping prevent held connections and stalled requests.
    • Requests cancelled during retry preparation now clean up any response still in use.
    • When retry capacity is unavailable, the previous response is preserved for handling.
  • Documentation
    • Updated adapter guidance to describe retry timing, response handling, and pacing behavior.

Readiness boxes ticked by maintainer @lidge-jun at integration (2026-11-18) after the exact-head review and CI described in the PR comments.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Nov 14, 2026
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

Automatic draft conversion failed. Please convert this pull request to a draft manually until every box above is ticked.

Hygiene

✅ Deterministic PR hygiene checks passed.

@github-actions
github-actions Bot marked this pull request as draft November 14, 2026 02:54
@coderabbitai

coderabbitai Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

No actionable comments were generated in the recent review.

Walkthrough

Google retries now cancel pending response bodies during pre-pacing preparation. The send flow adds a callback after budget reservation and before pacing acquisition. Tests cover retry timing, response ownership, budget outcomes, cancellation, and pacing cleanup.

Changes: Google retry pacing

Layer / File(s) Summary
Pre-pacing send hook
src/adapters/physical-send.ts, tests/adapters/physical-send.test.ts
The send flow invokes beforePacing after budget reservation and before pacing acquisition. beforeDispatch remains after pacing admission. Tests cover callback ordering and reservation cleanup when preparation throws or aborts.
Google retry response lifecycle
src/adapters/google-http.ts, tests/adapters/google/*, structure/adapters/registry.md, structure/providers-and-adapters.md, scripts/test-layout/layout-adapters.json, tests/fixtures/test-layout-expected.json
Google retries cancel pending response bodies during pre-pacing preparation. Budget-refused retries preserve the previous response, and remaining pending responses are cancelled when the function exits. Tests cover retry timing, response handling, budget outcomes, aborts, and pacing cleanup. The documentation describes the ordering and ownership behavior, and the test-layout files register the new tests.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related PRs

  • lidge-jun/opencodex#7819: Both PRs change Google retry decisions involving Retry-After and the retained upstream response.

Suggested reviewers: luvs01

Merge Risk: Minimal · previous review 1b586

Google retries under request pacing now release the previous response before waiting for a pacing slot, so backoff no longer holds a slot. The change is small, covered by targeted tests, and has no identified merge-blocking risk.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: Google retry handling now releases the superseded response before waiting for a pacing slot. This matches the raw summary (beforePacing callback, cancellation of the p…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: Google retry handling now releases the superseded response before waiting for a pacing slot. This matches the raw summary (beforePacing callback, cancellation of the prior response body before backoff) and the PR objectives. The title is specific, concise, and uses the conventional fix(google) prefix.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 18f69fd and 1b586b1.

Files selected for processing: 1

  • tests/fixtures/test-layout-expected.json

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@lidge-jun

Copy link
Copy Markdown
Owner

Merge-manager review: correct — the superseded response is released before the pacing wait, the backoff sleeps without holding a slot, budget refusals still surface Retry-After, and a throwing/aborted beforePacing returns the reserved budget. Tests (virtual time + loopback server) are meaningful. Ready to integrate once the readiness checklist is ticked on the head and CI is green. #8744 and #8745 register tests on the same layout lines, so whichever lands later needs a dev merge. Follow-up idea (not required here): src/adapters/command-code.ts:790 and src/adapters/mimo-free.ts:288 still cancel the previous response after pacing admission.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T06:43:25.899363Z b89c20d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions
github-actions Bot marked this pull request as draft November 18, 2026 16:36
@lidge-jun
lidge-jun force-pushed the codex/google-retry-release-lease-before-pacing branch from 3beda4d to b89c20d Compare November 19, 2026 04:16
@lidge-jun

Copy link
Copy Markdown
Owner

Maintainer integration record (lane L3, 2026-11-19).

Integrating exact head b89c20d1f732fb04372bb136a4c0d8641bba94da under the maintainer-integration path.

  • Review. The same gpt-6.2-sol reviewer covered fix(adapters): replay a Google 429 only when it carries retry evidence #8897 and every round of this PR. The final check at this head gave APPROVE: all three remerge diffs were empty, and the patch is unchanged from the approved 18f69fd.

  • Behavior. Google keeps the fix(adapters): replay a Google 429 only when it carries retry evidence #8897 retry-evidence gate and hard-quota precedence. It releases the superseded response before backoff and acquires the pacing slot afterward. A refused retry reservation preserves the original response and Retry-After, and a failure during preparation refunds the unused reservation.

  • CI and threads. Exact-head PR CI is green, and there are 0 unresolved threads. The owner-enrichment shard timeout seen earlier came from an unchanged PowerShell fixture under runner load.

  • History. Rebased onto dev because of a conflict in structure/adapters/registry.md. The rebase check by the same reviewer gave APPROVE: all nine files keep byte-identical added and removed lines, and authorship is preserved.

  • Union guards on the merge with current dev: file-size ratchet, test layout, Spanish localization, preset counts, typecheck, and structure:check all passed.

  • Files also changed on dev since the branch base: tests/fixtures/test-layout-expected.json

@lidge-jun
lidge-jun marked this pull request as ready for review November 19, 2026 06:38
@lidge-jun
lidge-jun merged commit ea24cbc into lidge-jun:dev Nov 19, 2026
57 of 61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants