Repository navigation
fix(link): drain Child link sockets before a restart stops the tunnel, and count only sockets Codex opened - #8658
Merged
Merged
Conversation
…, and count only sockets Codex opened
…, Codex drop and status-less close, and the counter notes
…ing upgrade reach its 503 whatever the Home last advertised
…in as its one exception
…ild-t6-5 # Conflicts: # scripts/test-layout/layout-clients.json
Contributor
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Contributor
|
✅ Deterministic PR hygiene checks passed. |
…the Child, as on the Home
…t closes both legs, counted once
…el spawned, start-up check done, link ended), not on microtask counts or a zero timer
… Home and its two timers
…eadable link id included
Owner
Author
|
Maintainer integration record for this squash merge.
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
3 tasks done
lidge-jun
added a commit
that referenced
this pull request
Nov 13, 2026
lidge-jun
added a commit
that referenced
this pull request
Nov 13, 2026
… turn, today and with T6's two proposed fixes tests/e2e-style/link-websocket-child-restart.test.ts restarts the real TypeScript Child between the two requests of a pinned-Codex turn, in the order the runtime follows since #8658: the tunnel gate's onStopping drains the relay (sockets close 1001, upgrades get 503), the old listener stays bound for the drain phase, stops (refused), and a replacement binds the same port with no advertisement remembered. Variants model lane T6's proposals before they exist: the listener closing when the drain starts, and the replacement treating "no advertisement since start" as "try" (the relay's advertised seam), and both. Each runs with stream_max_retries 3 and 5, for a 5.5 s and a 1 s restart. The cases assert only that the turn completes; the transport it ends on, the 503s, the replacement's 426s and Codex's retry notices are printed as one JSON line per case. The restart phases are wall-clock waits against an external process, by design. runGateCodex gains an optional stderr line callback.
This was referenced Nov 13, 2026
lidge-jun
added a commit
that referenced
this pull request
Nov 13, 2026
…assertion, the gate scenarios through the real Child, a Child restart measurement, and the Autobahn client conformance run (#8681) * test(link): record the attested digest of the pinned Codex for the link WebSocket release gate tests/fixtures/link-gate-codex.json now carries the sha256 of codex-rs built at openai/codex 88c58562580aa4abe2da6ce4d7676bf171682230 for x86_64-unknown-linux-gnu, and its note records who built it, when, with which toolchain, target and command. With this digest, a run of tests/e2e-style/link-websocket-release-gate.test.ts with that binary is labelled release evidence; any other binary stays a smoke check. The revision is unchanged. structure/remote-link.md no longer says the fixture records none, and states that the digest attests one Linux x86_64 build. Nothing opens the gate: linkWebsocketTransportReady() stays false. * test(link): assert the pinned Codex sends no Origin on its link WebSocket upgrade (U11) Design T4 U11 needed a release-gate test that Codex's WebSocket client sends no Origin header; until now the gate showed it only indirectly (the Home refuses an upgrade carrying one). The relay stand-in now records the Origin header of every upgrade Codex sends, before forwarding it, and the gate's drain scenario asserts that every upgrade of both sessions arrived without one. The release-gate labels, the codex exec runner and the fixture turns move unchanged into tests/helpers/link-gate-codex.ts, so the real-Child scenarios that follow label their runs with the same digest check. The gate's scenario bodies are unchanged; runCodex takes an optional stream_max_retries, default 3 as before. * test(link): hash and run the same copy of the pinned Codex in the release gate (#8668 review P2-1) The gate hashed OCX_LINK_GATE_CODEX relative to the working directory and then spawned the value as given, which Bun looks up on PATH for a bare name. With OCX_LINK_GATE_CODEX=codex, the attested bytes saved as ./codex and another codex on PATH, every title said "release gate" while the PATH binary ran; the file could also be replaced between the hash at module load and the spawns. The value is now resolved once to an executable regular file (anything else is refused and the scenarios are skipped), copied read-only into a scratch directory, and that copy is both hashed and spawned by absolute path (tests/helpers/link-gate-binary.ts). tests/e2e-style/link-gate-codex-binary.test.ts covers a bare name with another codex on PATH, a replacement after hashing, and refused values, with two small scripts instead of Codex; all three cases failed against the earlier behaviour. structure/remote-link.md describes the copy and says the smoke rule once, for each Codex scenario's title (P3-1). * test(link): name #8668 and the build directory in the pinned Codex attestation note The note now says the attestation was made in #8668, and lists another build directory among the things that may change the digest (rustc embeds absolute paths). Review P3-2. * test(link): make the pinned-Codex copy test portable and clean the copy up on every path Review of the previous commit: - tests/e2e-style/link-gate-codex-binary.test.ts used #!/bin/sh scripts and a colon-separated PATH, which the Windows shards cannot run. The stand-ins for Codex are now copies of the running Bun executable named codex (codex.exe on Windows); each prints its own process.execPath, so a run shows which file executed. PATH is joined with path.delimiter, and the execute-permission case is POSIX-only. Against the earlier behaviour the bare-name case reports the PATH copy (bin/codex) where the hashed file (work/codex) was expected. - The copy keeps the file's name, so an extension a platform needs stays. - The gate registers the scratch cleanup before copying, and removes the scratch directory at once when the value is refused or preparing it throws. * test(link): run the pinned-Codex release-gate scenarios through the real Child relay tests/e2e-style/link-websocket-child-gate.test.ts drives the pinned Codex through the real TypeScript Child listener and relay (lane T6) in front of a real Home, where the gate file uses a test-only stand-in. It is opt-in on OCX_LINK_GATE_CODEX and labelled exactly as the gate file. Scenarios: - the switch goes off under a running turn: both requests of the turn run on the Child's one socket, and the next session's upgrade is answered 426 by the Child (not_advertised) and completes over HTTP; - the drain past its deadline: the tool result's create is fenced and never reaches the upstream, and the same request completes over HTTP; - design 11.2-8 (a): with the Child's own 101 carrying the 8 MiB limit, a larger request goes over HTTP while the socket is open; - a Home restart mid-turn: the turn completes after Codex reconnects (logged: the cut request ends 502 on the socket and Codex completes on a new socket). It observes the Child's link WebSocket diagnostics and the Home's request-log rows. Which side closes a fenced socket first is a race between the Home and Codex exiting, so the close is asserted without its side. * test(link): prove through a TCP tap that the oversized request never went on the real Child's socket The real-Child 11.2-8 (a) scenario asserted only that a socket opened, that no upgrade fell back and that the Home saw one HTTP row; an oversized message the Child refused with 1009 before the Home, followed by an HTTP retry, would have passed it too. A TCP tap between Codex and the Child now records each connection's request head, the Child's answer head and the bytes Codex sent after its request head. The scenario asserts that the Child's 101 carries x-codex-websocket-max-message-bytes at the 8 MiB limit, that Codex wrote less than the limit on every upgraded connection, and that the whole body went on one POST connection of its own. Raising the Child's limit to 64 MiB makes Codex send 8,408,470 bytes on the socket and the scenario fail. * test(link): pin the 8 MiB the real Child must advertise to the design value, not the Child constant The 11.2-8 (a) scenario took its expected header value and its size threshold from LINK_CHILD_WS_LIMITS.localMessageBytes, the constant the Child itself uses, and matched the header as a substring. It now parses x-codex-websocket-max-message-bytes and requires exactly 8388608 (design T4 6.1), and uses that fixed value for the request size and the byte checks. * test(link): type the TCP tap chunks in the real-Child release-gate scenarios bun check reported the tap callbacks taking string | Buffer; they receive Buffers. No behaviour change. * test(link): measure a real Child restart under a running pinned-Codex turn, today and with T6's two proposed fixes tests/e2e-style/link-websocket-child-restart.test.ts restarts the real TypeScript Child between the two requests of a pinned-Codex turn, in the order the runtime follows since #8658: the tunnel gate's onStopping drains the relay (sockets close 1001, upgrades get 503), the old listener stays bound for the drain phase, stops (refused), and a replacement binds the same port with no advertisement remembered. Variants model lane T6's proposals before they exist: the listener closing when the drain starts, and the replacement treating "no advertisement since start" as "try" (the relay's advertised seam), and both. Each runs with stream_max_retries 3 and 5, for a 5.5 s and a 1 s restart. The cases assert only that the turn completes; the transport it ends on, the 503s, the replacement's 426s and Codex's retry notices are printed as one JSON line per case. The restart phases are wall-clock waits against an external process, by design. runGateCodex gains an optional stderr line callback. * docs(link): say that only the Codex scenarios are skipped without the pinned binary (#8668 review P3-3) structure/remote-link.md and the gate file comment said every case is skipped when OCX_LINK_GATE_CODEX is absent; the pin test always runs. * test(link): give the codex on PATH other bytes, so the digest tells it from the hashed file (#8668 review P3-4) Both stand-ins were copies of the same Bun executable, so the digest assertion could not tell work/codex from bin/codex. The PATH-side file now holds other bytes (mode 0755; the gate must never run it), and the test asserts the digest differs from its hash. Against c28e59a behaviour the bare name runs that PATH file (ENOEXEC), a replaced file is what runs (ENOEXEC), and a directory throws EISDIR while hashing; a 0644 file was already refused. * test(link): reflink the pinned-Codex copy where the file system can (#8668 review P3-5) The copy now uses COPYFILE_EXCL | COPYFILE_FICLONE, so the 1.6 GB binary costs no space on APFS, btrfs or XFS, and an interrupted direct run leaves nothing large behind there. Checked on Bun 1.4.4: with FICLONE the exclusive flag still refuses an existing destination (EEXIST) and keeps it; on ext4 the copy falls back to an ordinary one. A new case makes the first copy writable and requires EEXIST for a second copy into the same scratch directory; without COPYFILE_EXCL it fails. * docs(link): describe the Child restart measurement in the release-gate paragraph * test(link): mark the restart measurement turn when Codex actually exits The codex exit mark was written after the replacement Child was bound and the run awaited, so a turn that ended early (over HTTP through the draining Child) had its boundary at the bind. It is now marked when the process exits, and the first-session counts stop there. * test(link): mark the restart turn at the Codex process exit itself, not after its output is collected runGateCodex resolves once the process exited and its stdout and stderr were read; the restart measurement marked the exit there, which output draining could delay. runGateCodex gains an onExit callback fired from child.exited, and the measurement uses it to bound the first session. * test(link): the Autobahn fuzzingserver client conformance run for the Child link WebSocket leg (design 3.3) tests/clients/link-ws-autobahn.test.ts drives every fuzzingserver case through the Child's own 101 check and a real HomeLeg, with an echo stub in place of Codex that sends one message at a time as Codex does. It is opt-in on a running fuzzingserver (OCX_LINK_WS_AUTOBAHN_URL) and its report directory (OCX_LINK_WS_AUTOBAHN_REPORTS), and it judges each case against reviewed rules in tests/fixtures/link-ws-autobahn-expectations.json: deflate cases unimplemented (the Child offers no extension), a Home message over 4 MiB + 64 KiB or over 4,096 data frames closed 1009, other binary messages closed 1003, everything else OK, NON-STRICT or INFORMATIONAL. Against crossbario/autobahn-testsuite@sha256:519915fb...3074 all 517 cases land in their class. * test(link): make the Autobahn harness enforce every condition its verdict rests on Review of 559c34c: a refusal (1003, 1009) now also requires, from the case report, that the Child sent nothing but a close frame and that the close was clean with no handshake or drop timeout; the deflate cases require UNIMPLEMENTED with a clean OK close; each run uses its own agent name, so a report left in the directory by an earlier run cannot supply the verdict, and a control exchange that does not close 1000 fails the run; setting only one of the two variables fails and names the other; the run prints the expectations sha256, the agent and the server. * test(link): accept the fuzzingserver control close without a status code (1005) The fuzzingserver ends getCaseCount and updateReports with a close frame that carries no code, which Bun reports as 1005; a dropped connection (1006) or any other code still fails the run. * test(link): fail an Autobahn refusal case whose report has no valid frame statistics A missing or malformed rxFrameStats was read as no frames, which would pass the no-data check; it now fails the case, and a refusal must also show the Child's close frame. * test(link): commit the recorded Autobahn run and refuse a fuzzingserver off loopback (#8681 review P2-1) structure/remote-link.md described a pinned image and a loopback binding that nothing in the tree started or checked. tests/fixtures/link-ws-autobahn/ now holds the fuzzingserver spec (every case) and run.sh, the recorded run: it starts crossbario/autobahn-testsuite@sha256:519915fb...3074 with that spec, publishes it on 127.0.0.1:9001 only, waits for it, runs the test, removes the container and writes provenance (commit, clean tree, Bun, image, expectations and spec sha256, command, exit) and the report tarball with its sha256. The test refuses an OCX_LINK_WS_AUTOBAHN_URL whose host is not loopback, and the structure sentence cites the script. * test(link): pin the Autobahn case set (#8681 review P3-1) The expectations file records caseCount 517 and the run asserts the server reports exactly that many cases, and that every rule matches at least one case of the run, so a different suite or a stale rule fails instead of passing on a subset. * test(link): require a clean close in every Autobahn class and check the stated limits against the Child (#8681 review P3-2) The comment said the clean close is checked for every class, but the pass class did not use it; it now does. The expectations file states the Home message and fragment limits its 1009 rules are written for; the run now asserts they equal LINK_CHILD_WS_LIMITS. * test(link): keep the pinned-Codex copy for the whole process and remove it at exit (#8681 review P3-3) The copy lives once per module registry. Each file removed it from its afterAll, so a plain multi-file bun test (one shared registry) removed it under the next file, whose scenarios then failed with spawn ENOENT while titled release gate (reproduced: real-Child file then gate file, 7 failures). The helper now removes it once, when the process exits, and at once when the value is refused or preparing it throws; the per-file afterAll calls are gone. The same two files in one run now pass 12 of 12, and a plain process that loaded the helper leaves no copy behind. * test(link): title the real-Child drain scenarios with what they assert (#8681 review P3-4) The titles said the idle socket is fenced and the next create is fenced on the socket, but the assertions accept a close from either side, and Codex closed first in every recorded run. The titles now say what is asserted: the turn finishes on the socket (or its tool result never reaches the upstream over it), the socket closes from either side, and the request completes over HTTP. The fence itself is shown by the stand-in scenarios in the gate file. * test(link): require exactly one socket in the real-Child 11.2-8 (a) scenario (#8681 review P3-5) At least one upgraded connection allowed a loop in which Codex ignores the advertised limit, is cut under 8 MiB per attempt and ends on HTTP with every assertion passing. The scenario now requires exactly one upgraded connection through the tap and exactly one link_ws_open record. * test(link): end the waits on Codex exiting first, and say where the scenarios wait on time (#8681 review P3-6) The restart measurement waited for the tool result to reach the Home, and the real-Child Home restart for the first request, without racing Codex exiting: a binary that fails early cost the 180 s bound per case, with no stderr. untilOrExit (tests/helpers/link-gate-codex.ts) ends such a wait at once with Codex exit code and the tail of its stderr, and leaves nothing behind when Codex exits after the event. With a fake codex that exits 3 at start, both cases now fail within a second naming it. The helper header no longer says the scenarios never sleep: the restart phases are wall-clock waits by design. * docs(link): say what a Child restart drain does to the Codex session, as measured (#8681 review P3-7) structure/remote-link.md said the drain 503 lets Codex keep its WebSocket session for the replacement instead of falling back to HTTP. With the pinned Codex every 503, like a refused or dropped connect, spends one stream retry, and the four today cases of the restart measurement ended the interrupted turn on HTTP. Only that clause of the sentence changes. * test(link): end the stand-in Home restart wait on Codex exiting first Review of G-2-3: the gate file Home restart scenario still awaited the first request without racing Codex exiting. It now uses untilOrExit, as the helper header says every wait on an event does; with a fake codex that exits 3 at start it fails within a second naming its stderr. * test(link): end untilOrExit when the Codex run itself rejects before the event Review of G-2-3: a rejected run (its output could not be read) was swallowed, so the wait held until the test bound. It now rejects the wait when it comes before the event, and is still handled after it. Checked with three probe cases (rejecting run, early exit, late settle): all pass with no unhandled rejection. * test(link): describe the real-Child drain cases by what they observe, not by a fence they do not establish Review of G-2-3: comments in the real-Child file still said the create was fenced; the assertions accept a close from either side and establish only the upstream and transport outcomes. The comments now say so and point to the stand-in scenario for the fence. * test(link): connect to an IPv6 loopback fuzzingserver without the URL brackets Review of G-2-3: the loopback check accepts http://[::1]:9001, whose URL.hostname keeps the brackets, which node:net cannot connect to. The case runner now strips them. The recorded runs use 127.0.0.1 and are unaffected. * test(link): name try-when-unknown as the Child's rule since #8682 in the restart measurement; the old rule stays for comparison * docs(structure): the restart measurement names try-when-unknown as the Child's rule since #8682 * test(link): run try-when-unknown through the relay's default rule in the restart measurement; count the replacement's answers around its fetch * test(link): say what --isolate does to the pinned Codex copy * test(link): run.sh refuses a used run directory and stops when the fuzzingserver never answers * docs(structure): the Autobahn run.sh is the recording script * test(link): the real-Child switch case says the 426 is answered without forwarding the upgrade * docs(structure): the measured restart outcomes are from one Linux x86_64 machine, printed, not asserted * docs(devlog): gate-opening item 4: #8681 already runs the restart scenarios under #8682 rule, through test seams
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the Child side of the Responses WebSocket over a Remote Link (#8614, #8619, #8620). The final reviews of #8619 and #8620 left items that must be fixed before the Child's release gate opens; this PR fixes them. The gate stays closed, so nothing changes for anyone today.
LinkTunnelGatehas an optionalonStoppinghook. The supervisor calls it whenstop()begins, before it signals the tunnel, and the relay drains: open link sockets close with 1001 on both legs, setups are aborted, and every upgrade from then on, new or still in setup, gets the transient 503 (link_websocket_unavailable, counted asdraining). The supervisor also calls it when the link ends (its sidecar is gone or names another link) and the runtime is about to recycle. The hook lives in the supervisor rather than inruntime.ts, so every path that stops the supervisor (restart, recycle, signal) drains first. While draining, an upgrade reaches its 503 even if the Home's last answer turned the advertisement off; a closed Child gate or the Child's opt-out still gives today's 426 with no network I/O. Eachserve()builds a new listener and supervisor, so a drained relay is never reused.server.upgradereturned false, the Child countedcloses.stopped, wrotelink_ws_close:stoppedand counted a complete request in the traffic book for a socket Codex never had. When it threw, the leg stayed in the set with a paused Home socket. Both now go throughHomeLeg.abandon(): the Home leg closes with 1001, only thehandshakefallback is counted, and the slot is released once the Home connection closes. A socket's traffic flow now begins when Codex's socket opens.HomeLegarms the 5 s close bound when it is created and stops itself ifattachhas not run by then. Scratch runs on the pinned Bun 1.4.4 show that Bun runsopensynchronously insideserver.upgradewhen it returns true, and thatserver.upgradereturns false (withreq.signalaborted) if Codex left before the call. The bound is therefore a guard for runtimes that behave differently.server.upgradethrew after runningopen) closes both legs as a stop does and counts the socket once; it used to close only the Home leg.LinkWebsocketCountersJSDoc andstructure/remote-link.mdnow say that a non-localOriginrefused with 403 is not counted, and thatfromCodexBytescounts text messages arriving while the Home leg is open. The structure page also explains why the Child's?transport=1shape differs from the Home's per-rowws: the Home counts what it did with upgrades per link (refused), while a Child serves one link and itsfallbacksare the attempts it sent back to HTTP or told to retry, with reasons only a Child sees. The name is kept.What keeps it inert:
linkWebsocketChildReady()is still false, so the ingress answers today's 426 and the relay never runs. The relay subscribes to the supervisor's stopping hook in every supervised link-mode listener; with the gate closed, the hook flips two flags and walks two empty sets: no I/O, no counting, no diagnostics.Verification
Tests that failed first (
tests/clients/link-ws-relay-edges.test.ts):server.upgradereturning false answers 426, counts no close and no request, and frees the slot. Failed:link_ws_close:stoppedwas recorded and a request was counted.server.upgradethrowing does the same and leaves no leg behind. Failed: the leg stayed.stop()notified.Final-review fixes (failed first unless marked):
incomplete: 1, the 1005 halfincomplete: 0. Failed: the drop counted complete.server.upgradethrowing after Bun opened Codex's socket closes both legs and counts the socket once. Failed: Codex's socket was never closed.onInitializedtest seam inClientLinkSupervisorDeps; link ended) instead of microtask counts and a zero timer. Test-only.Existing tests updated:
tests/clients/link-ws-relay.test.ts(the slot-release case awaits the newonReleasedseam instead of retrying until a 101; the stop-in-hold case asserts exactly 426; the oversized-message case asserts that the Home gets 1011; two directHomeLegcases take a flow factory and mark the socket upgraded, as the relay does) andtests/clients/link-ws-child-status.test.ts(waits on the Child's own release before readingopen: 0).Commands (exit codes) on 03a7cf0 (the T6-5-1 head 8494612 plus one commit per final-review item; it merges cleanly with dev 6f58f62):
bun install --frozen-lockfile: exit 0bun run typecheck: exit 0bun run privacy:scan: exit 0bun run structure:check: exit 0bun scripts/test.ts --parallel=2over tests/clients/link-ws-relay, link-ws-relay-edges, link-ws-child-status, link-ws-fallback, link-ws-transport, link-ws-framer, link-ws-advertisement, link-ws-activity-holds, client-link-tunnel, client-link-runtime, client-machine-listener, client-link-relay, link-relay-bound-transport, link-boundary, tests/e2e-style/link-ws-child-home, tests/e2e-style/link-websocket-release-gate, tests/lab/core-lab-boundary, tests/test-layout, test-layout-tooling, ci-workflows/structure-ssot, ci-workflows/file-size-ratchet: exit 0 (384 pass, 8 skip, 0 fail; the skips are seven release-gate cases that need the pinned codex-rs binary and one macOS-only case)Review: a gpt-6.1-terra reviewer (xhigh, read-only, its own worktree) reviewed the whole slice. Round 1: no P0-P2; two P3s (the link-ending path did not drain; the ingress's advertisement check could bypass a drain), both accepted and fixed with the tests above, which failed first. Round 2: no P0-P2; one documentation P3 (the structure page's earlier 426 sentence did not name the drain), fixed in cb79c1f. The coordinator's fresh final review of 8494612 found one P2 (a dropped Codex counted complete) and five P3s; each is fixed in its own commit, and a gpt-6.1-terra round on those commits found none. Open P0-P2: 0.
NOT RUN: the full
bun run testsuite (4-core VM rule: focused files only); macOS/Windows; the release-gate cases intests/e2e-style/link-websocket-release-gate.test.tsthat need the pinned codex-rs binary (OCX_LINK_GATE_CODEX, absent here; they skip).Checklist