Skip to content

test(link): release-gate harness for the link WebSocket: a no-Origin assertion, the gate scenarios through the real Child, a Child restart measurement, and the Autobahn client conformance run - #8681

Merged
lidge-jun merged 50 commits into
devfrom
test/link-gate-evidence
Nov 13, 2026

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

A release-gate harness for the link WebSocket (design T4 11.1 item 13 and 11.2), built on the pinned Codex digest from #8668: the tests, fixtures and run script below. The results quoted in this description are measurements taken with it on the build VM; they are not part of the diff. All of it is test-only and opt-in like the gate itself, with no change under src/. The gate stays closed: linkWebsocketTransportReady() and linkWebsocketChildReady() are still false.

The branch history includes #8668's commits from before its squash; their content is already on dev, so this PR's diff is only the work below.

1. U11 and the scenarios through the real Child

U11: Codex sends no Origin. Design T4 U11 asks for a release-gate test that Codex's WebSocket client sends no Origin header. Until now the gate showed it only indirectly: the Home refuses an upgrade that carries one. The relay stand-in (tests/fixtures/link-child-relay-standin.ts) now records the Origin header of every upgrade Codex sends, before forwarding it. The gate's drain scenario asserts that every upgrade of both its sessions arrived without one. With the assertion flipped, that scenario fails; with it as written, it passes.

The same commit moves the release labels, the codex exec runner and the fixture turns out of the gate file, unchanged, into tests/helpers/link-gate-codex.ts. That way the second file labels its runs with the same digest check. The gate's scenario bodies are unchanged. Since the merges from #8668, that helper also owns the read-only scratch copy of the binary, which is what gets hashed and spawned.

The release-gate scenarios through the real Child. tests/e2e-style/link-websocket-child-gate.test.ts drives the pinned Codex through the real TypeScript Child listener and relay (lane T6) in front of a real in-process Home. The gate file uses a stand-in instead. Each scenario reads the Child's link WebSocket diagnostics and the Home's request-log rows. While the switch is on, a link row carries the transport its request arrived on. Once a scenario switches it off, the Home writes no link field, so the row is "unmarked"; with no new socket opened by the Child, it is a request relayed over HTTP.

  1. The switch goes off under a running turn (titled with what is asserted: the turn finishes on the socket, the socket closes from either side, and the next session ends on HTTP). Both requests of the turn, the tool result included, run on the Child's one socket (websocket:200 twice). In the next session, the Child's preflight learns that the Home stopped advertising. The Child then answers 426 without forwarding the upgrade to the Home (link_ws_fallback:not_advertised), and the request completes over HTTP.
  2. The drain past its deadline (titled likewise: the tool result never reaches the upstream over the socket, the socket closes from either side, and the request completes over HTTP). The tool result never reaches the upstream over the socket (two upstream requests in total). Codex's reconnect gets the Child's 426 (not_advertised), and the same request completes over HTTP.
  3. Design 11.2-8 (a). A TCP tap between Codex and the Child records every connection's request head, the Child's answer head, and the bytes Codex sent after its request head. The Child's 101 carries x-codex-websocket-max-message-bytes, and the parsed value must be exactly 8388608. That is the design's 8 MiB (T4 6.1), fixed in the test independently of the Child's constant, and the same fixed value is the threshold for the size checks. There is exactly one upgraded connection and exactly one link_ws_open. On that connection Codex wrote less than 8 MiB, so the larger request never went on the socket. The whole body went on a POST connection of its own, and the Home saw one HTTP row. Two mutations, not committed, are kept with the run outputs. Raising the Child's limit to 64 MiB fails the header check. With the header check also removed, Codex wrote 8,408,470 bytes on the socket and the byte check failed.
  4. A Home restart mid-turn. The turn completes after Codex reconnects. In each of the runs below, the cut request ended websocket:502 on its socket and Codex completed on a new socket (websocket:200). The test logs this but does not assert the transport, which is Codex's own retry policy.

What differs from the stand-in. The real Child records which side closed a socket. Where its records were printed during development (once per drain scenario), it was Codex that closed first (link_ws_close:client). These tests therefore assert that the close happened, not which side made it, and do not establish that the Home fenced the socket. The gate file's stand-in scenario shows the fence itself. After the switch goes off, the Child's refusal comes from its own preflight (not_advertised), not from a Home refusal (home_refused).

Shared helper. tests/helpers/link-gate-codex.ts holds the gate's labels, the Codex runner and the fixture turns, moved unchanged from the gate file. It also holds #8668's read-only copy of the binary. The copy belongs to the process and is removed when the process exits, so the files can run together in one plain bun test (shared module registry). With --isolate (the repository's runner; one process, a fresh global per file) each file makes its own copy, removed at exit; a run killed by a signal leaves it. Waits on an event, such as the first request or the tool result reaching the Home, end at once with Codex's stderr if Codex exits first, or with the run's own error if it fails (untilOrExit). This covers the gate file's stand-in Home restart as well.

2. A Child restart under a running turn

What it measures. tests/e2e-style/link-websocket-child-restart.test.ts restarts the real TypeScript Child between the two requests of a pinned-Codex turn (the tool result is in flight). It follows the order the runtime uses since #8658:

  1. The tunnel gate's onStopping drains the relay: open sockets close 1001, and every upgrade gets the transient 503 (draining).
  2. The old listener stays bound for the drain phase, then stops, so connections are refused.
  3. A replacement binds the same port, with no advertisement remembered, as a restarted process would.

Variants model the two Child changes proposed by lane T6, through the relay's test seams. One of them is now the product's rule: since #8682 (4aa5dc40b, merged here), the TypeScript Child tries the upgrade when it has no answer from its Home since start, behind the closed gate. The rule before #8682 is kept for comparison, and closing the listener at drain start is still only proposed:

Each variant runs with Codex's stream_max_retries 3 (the gate's setting) and 5 (Codex's default), for a restart of 5.5 s (the supervisor's 5 s TERM bound plus a 0.5 s bind) and of 1 s. The cases assert only that the turn completes. Per case they print the timeline: every 503, every judgement the replacement makes, Codex's retry and fallback notices, the transport that completed the turn, and the next session's transport. The restart phases are wall-clock waits against an external process, by design. runGateCodex (tests/helpers/link-gate-codex.ts) gains two optional callbacks for this: one for each stderr line, and one for the process exit.

Results. Three runs of all 16 cases on 172432b05, all passing (73.6, 73.8 and 74.2 s). The transports and counts agree across the three runs and with the earlier runs on c827e8c62, cea31ad92 and cdab213df, and times vary by under 0.5 s. Run 1:

Restart retries Variant 503 phase refused 503s (turn) 426s from the new Child (turn) new bound at turn completed at Codex exited at turn completed on next session
5.5 s 3 try-when-unknown 5.0 s 0.5 s 3 0 5.50 s 1.47 s 1.62 s http websocket
5.5 s 3 old-rule 5.0 s 0.5 s 3 0 5.50 s 1.47 s 1.61 s http http
5.5 s 3 try-when-unknown+close-at-drain 0.0 s 5.5 s 0 0 5.50 s 6.63 s 6.75 s http websocket
5.5 s 3 old-rule+close-at-drain 0.0 s 5.5 s 0 0 5.51 s 6.73 s 6.85 s http websocket
5.5 s 5 try-when-unknown 5.0 s 0.5 s 4 0 5.50 s 6.53 s 6.64 s websocket websocket
5.5 s 5 old-rule 5.0 s 0.5 s 4 1 5.51 s 5.85 s 5.97 s http websocket
5.5 s 5 try-when-unknown+close-at-drain 0.0 s 5.5 s 0 0 5.51 s 6.13 s 6.25 s websocket websocket
5.5 s 5 old-rule+close-at-drain 0.0 s 5.5 s 0 1 5.51 s 6.08 s 6.21 s http websocket
1.0 s 3 try-when-unknown 0.5 s 0.5 s 1 0 1.01 s 1.40 s 1.53 s websocket websocket
1.0 s 3 old-rule 0.5 s 0.5 s 1 1 1.00 s 1.49 s 1.61 s http websocket
1.0 s 3 try-when-unknown+close-at-drain 0.0 s 1.0 s 0 0 1.00 s 1.43 s 1.55 s websocket websocket
1.0 s 3 old-rule+close-at-drain 0.0 s 1.0 s 0 1 1.01 s 1.43 s 1.56 s http websocket
1.0 s 5 try-when-unknown 0.5 s 0.5 s 1 0 1.01 s 1.45 s 1.57 s websocket websocket
1.0 s 5 old-rule 0.5 s 0.5 s 1 1 1.01 s 1.40 s 1.53 s http websocket
1.0 s 5 try-when-unknown+close-at-drain 0.0 s 1.0 s 0 0 1.01 s 1.47 s 1.60 s websocket websocket
1.0 s 5 old-rule+close-at-drain 0.0 s 1.0 s 0 1 1.01 s 1.48 s 1.61 s http websocket

The replacement's answers are counted around the listener's own fetch handler (a serve wrapper that marks every upgrade request with its answer: 101, 426 or 503), the same way for both rules. "503s (turn)" and "426s from the new Child (turn)" count only up to the moment the turn's Codex process exited, which is marked from the process's own exit (before its output is collected; runGateCodex's new onExit). "Next session" is a second codex exec started once the replacement is bound.

What the numbers show. Three outcomes are measured separately: the transport the interrupted turn ends on, when it completes, and the transport of the next session.

  • The interrupted turn's transport. With stream_max_retries 3, a mid-turn cut leaves about 1.4 s of WebSocket retries, and every 5.5 s restart ends the turn on HTTP, whatever the variant. With stream_max_retries 5, and in every 1 s restart, Codex's next attempt reaches the replacement within its budget. Under the old rule the replacement then answers 426 (not advertised) and the turn ends on HTTP; under try-when-unknown, dev's rule since feat(link): try the WebSocket upgrade when the Child has no answer from its Home yet (closed gate) #8682, the turn ends on the WebSocket, with or without close-at-drain. Closing the listener at drain start changes no turn's transport: a refused WebSocket connect spends Codex's retry budget like a 503, because pinned codex-rs maps the connect I/O error to a stream error.
  • When the turn completes. Both rules without close-at-drain, at 5.5 s with retries 3 complete at about 1.4–1.5 s, over HTTP through the still-draining old Child: its HTTP relay keeps working while upgrades get 503. That is a limit of the model: the stand-in tunnel stays connected during the drain. In the real runtime the supervisor signals the tunnel during that window, and the old Child's HTTP relay may then answer 503 home_unreachable with Retry-After: 1 instead. No case here passes through that window. With the listener closed at drain start, that HTTP request is refused too. Codex then waits for the network (5 s) and the turn completes at about 6.6–6.7 s, after the replacement binds.
  • The next session's transport. WebSocket in every case but one: the old rule at 5.5 s with retries 3, where the next session also ends on HTTP. In that case the turn completed through the old Child, so the replacement never saw a relayed request, kept no advertisement, and answered the next session 426. Closing the listener at drain start avoids that in this case because the turn's own HTTP request goes through the replacement and teaches it the advertisement. try-when-unknown avoids it by trying the Home, so on dev since feat(link): try the WebSocket upgrade when the Child has no answer from its Home yet (closed gate) #8682 the next session uses the WebSocket in every measured case.

Codex's retry behaviour at the pin (evidence for the restart results, and where it contradicts design C6)

Open item for the gate-opening change: design C6 says a transport-level ConnectionFailed retries without limit and without a transport switch. For the WebSocket connect, that is not what the pinned codex-rs does: a refused WebSocket connect spends stream_max_retries like a 503 and then switches the session to HTTP. Only an HTTP request gets the unlimited wait. The measurement:

Measured 2026-11-13 08:01-08:03 UTC by lane G with a scratch probe outside this repository (raw logs kept on the build VM). A TCP front on a fixed loopback port forwards to a fixture provider. The turn's second request (the tool result) is cut when it arrives, as a Child restart would cut it, and an outage starts. Codex config as in the gate: wire_api responses, supports_websockets true, request_max_retries 1, stream_max_retries 3 or 5. One run per row.

Results (times from the cut)

Outage retries What Codex did Session ends on Completed
upgrades answered 503 3 503 at 0.20, 0.61, 1.36 s; switched to HTTP at 1.36 s HTTP, POST at 1.39 s yes
upgrades answered 503 5 503 at 0.20, 0.58, 1.35, 3.05, 6.06 s; switched at 6.06 s HTTP, POST at 6.09 s yes
upgrade accepted, closed before the 101 3 attempts at 0.21, 0.63, 1.38 s ("Handshake not finished"); switched at 1.38 s HTTP, POST at 1.41 s yes
upgrade accepted, closed before the 101 5 attempts at 0.19, 0.60, 1.42, 3.10, 6.06 s; switched at 6.06 s HTTP, POST at 6.09 s yes
listener closed (refused), 12 s, session on WebSocket 3 retries 2/3 at 0.19 s and 3/3 at 0.58 s; switched to HTTP at 1.42 s ("Connection refused (os error 111)"); then the refused POST waits "for network" at 1.65 s and 6.90 s HTTP, first POST 4.9 s after the rebind (16.9 s) yes
refused, 12 s, WebSocket 5 retries at 0.22, 0.59, 1.36, 2.83 s; switched at 5.72 s; refused POST waits at 5.94 and 11.18 s HTTP, first POST 9.2 s after the rebind (21.2 s) yes
refused, 3 s, WebSocket 3 retries at 0.21 and 0.61 s; switched at 1.39 s; refused POST waits at 1.64 s HTTP, POST 3.7 s after the rebind (6.67 s) yes
refused, 3 s, WebSocket 5 retries at 0.21, 0.63, 1.48 s; the listener came back at 3.00 s WebSocket: a new upgrade 0.20 s after the rebind, completed on it yes
refused, 12 s, session already on HTTP (first upgrade answered 426) 3 and 5 the refused POST waits "for network" at 0.19 s and 5.44 s; no turn failure HTTP, POST 3.5 s after the rebind (15.47 s) yes

The three facts

  1. A refused WebSocket connect does not retry without limit. It spends stream_max_retries exactly like a 503, and with the budget spent Codex switches the session to HTTP. Design C6 (unbounded ConnectionFailed retries without a transport switch) holds for HTTP, not for the WebSocket connect. Source at the pin: an I/O error at the WebSocket connect maps to TransportError::Network (codex-api/src/endpoint/responses_websocket.rs:722, map_ws_error), which becomes CodexErr::Stream (codex-api/src/api_bridge.rs:324) and so takes the Backoff/budget path. An HTTP connect error maps to TransportError::Connection, which becomes CodexErr::ConnectionFailed (api_bridge.rs:321-322) and takes the unbounded "Reconnecting... waiting for network" path (core/src/responses_retry.rs:94-119). A refused plain HTTP request in the same window is retried without limit (5 s, then about 5.2-5.3 s, then 10 s; up to 60 s) and does not fail the turn.
  2. Time from the bind to Codex's reconnect is the time to Codex's next scheduled attempt:
    • While still within the WebSocket budget, attempts come at about 0.2, 0.6, 1.4, 3.0 and 6.0 s after the cut (200 ms doubling, ±10 %). After a 3 s outage with retries 5, the reconnect came 0.20 s after the bind, on WebSocket.
    • Once the session is on HTTP, the waits are 5 s, then 10 s, so after a 12 s outage the first request came 3.5-9.2 s after the bind.
    • The first retry after the cut is Codex's retry 1. Its notice is hidden, so the log starts at "2/N". A mid-turn cut therefore leaves N-1 reconnect attempts.
  3. An upgrade accepted and closed before the 101 ("WebSocket protocol error: Handshake not finished") counts against the 503-style budget, not like a refused connect, and ends the session on HTTP when the budget is spent.

What follows for the restart

With stream_max_retries 3, the WebSocket budget after a mid-turn cut lasts about 1.4 s; with 5, about 6.0 s. Within that window it makes no difference whether a restarting Child answers 503, closes the listener (refused) or accepts and drops the upgrade: each attempt spends one retry, and a restart outlasting the window ends that Codex session on HTTP. So closing the listener when the drain starts (T6 fix 1) does not give unbounded retries by itself. It turns 503s into refusals, and both count the same. Only an HTTP-already session gets the unbounded wait. The real-Child measurement in this PR (above) takes it from there.

3. The Autobahn fuzzingserver client conformance run (design 3.3, R2-06)

This is one of the two conditions for opening the Child's release gate (src/client/link-websocket-limits.ts). tests/clients/link-ws-autobahn.test.ts drives every fuzzingserver case through the code the Child runs against a Home: its own 101 check (validLinkUpgradeAnswer) and a real HomeLeg (frame reader, masked client frames, ping answers, close handshake, message and fragment limits) on a node:net socket. In place of Codex, an echo stub sends each text message back through leg.fromCodex, one at a time, as Codex sends one request at a time. The test is opt-in on OCX_LINK_WS_AUTOBAHN_URL and OCX_LINK_WS_AUTOBAHN_REPORTS; setting only one of them fails and names the other. Each run reports under its own agent name, so a report left in the directory by an earlier run cannot supply the verdict.

Every case is judged against reviewed rules in tests/fixtures/link-ws-autobahn-expectations.json. The first matching rule decides, and each rule reads the case's own report:

Cases Required Why
12.*, 13.* (permessage-deflate) UNIMPLEMENTED, close OK, clean close the Child offers no extension
9.1.5, 9.1.6 close 1009, no data frame from the Child, clean close an 8 MiB / 16 MiB text message is over the 4 MiB + 64 KiB Home message limit
9.2.5, 9.2.6 as above, 1009 an 8 MiB / 16 MiB binary message: the size limit refuses it before its type is acted on
9.3.1–9.3.3, 9.4.1–9.4.3 as above, 1009 a 4 MiB message in 64 B, 256 B and 1 KiB fragments is 65,537, 16,385 and 4,097 data frames (the server's own frame counts): over 4,096
1.2.*, 9.2.*, 9.4.*, 9.6.*, 9.8.* (the rest) close 1003, no data frame from the Child, clean close binary messages within the limits are refused, never echoed
every other case behavior OK, NON-STRICT or INFORMATIONAL; close OK or INFORMATIONAL

Autobahn itself reports the 1003 and 1009 cases as FAILED, because it expected an echo; the rules above are what the Child's design requires instead. The recording script is in the tree (its outputs are not): tests/fixtures/link-ws-autobahn/run.sh starts crossbario/autobahn-testsuite@sha256:519915fb568b04c9383f70a1c405ae3ff44ab9e35835b085239c258b6fac3074 with the spec beside it (every case). It publishes the server on 127.0.0.1:9001 only, runs the test, removes the container, and writes provenance (it refuses a run directory that is not empty, and stops with a message if the server has not answered within 20 s): commit, clean tree, Bun version, image, expectations and spec sha256, command and exit. It also writes the report tarball and its sha256. The test refuses a server URL whose host is not loopback (127.0.0.1, localhost or [::1], the last connected without its brackets). It asserts that the server reports exactly the reviewed case count (517), that every rule matches at least one case, that the expectations file's limits equal the Child's LINK_CHILD_WS_LIMITS, and that every class closed cleanly. Results are under Verification.

Two findings came out of building it, neither a defect:

  • The fragment limit counts data frames. Autobahn's 1 KiB case is 4,097 frames (a first frame plus 4,096 continuations) and is refused, while 4,096 frames are accepted.
  • A binary message that is also over a limit gets 1009 rather than 1003, because the limit is checked before the payload is held.

A correction to structure/remote-link.md. The drain sentence of the TypeScript Child paragraph (from #8658, rewritten by #8679) now carries both: #8679's account of the retry budget and this measurement's per-case outcomes, with 3 named as the gate's setting and 5 as Codex's default. The final sentence is quoted under Verification.

Not measured

The 9.2 controlled validation and U15 (the largest upstream chunk on the canonical upstream) were skipped by the owner on 2026-11-13 and were not measured. Neither passed; neither was run. The Child's transport-only RSS acceptance is built next, on its own branch, against the Child relay wait observer from #8680 and #8683. The Home's RSS acceptance is run separately and recorded with the gate-opening change.

Verification

Final head 20a0bd812. It changes one phrase of structure/remote-link.md after 172432b05, where the runs below were made (the P3-5 qualifier now reads "measured on one Linux x86_64 machine"); bun run structure:check, tests/ci-workflows/structure-ssot.test.ts (51 pass) and bun run privacy:scan were rerun on 20a0bd812, exit 0 each.

The dev merge is 1dbffa43e: dev 726bd713a (#8679) merged with a normal merge commit, on top of the earlier merges of 4aa5dc40b (#8682) and 23a7e18e7 (#8680). The only conflict was the drain sentence of the TypeScript Child paragraph in structure/remote-link.md; git show --remerge-diff 1dbffa43e touches only that paragraph. The resolution keeps #8679's account of the retry budget and this branch's measured outcomes, and names 3 as the gate's setting and 5 as Codex's default. The final sentence:

A Child restart drains earlier: the tunnel supervisor tells the relay when its stop() begins, or when the link ended (its sidecar gone or naming another link, or the connected link id unreadable for several checks in a row) and the runtime is about to recycle, before it signals the tunnel, and from then on open link sockets close with 1001 on both legs and every upgrade, new or still in setup, gets the transient 503 (counted as draining; while draining the Home's last advertisement no longer sends an upgrade to the 426, though a closed Child gate or the Child's opt-out still does), so Codex retries on its WebSocket session instead of falling back to HTTP at once; each 503 spends one of Codex's stream retries, as a refused or dropped connect does, and so does each refused connection while no listener is bound (about 1.4 s at stream_max_retries 3, the gate's setting; Codex's default is 5; measured with the pinned codex-rs 88c5856; see devlog/_plan/261113_link_websocket_gate_opening/000_plan.md, item 3), so the session stays on the WebSocket only when the replacement accepts the upgrade within that budget, and otherwise moves to HTTP for the rest of it (measured with the pinned Codex in tests/e2e-style/link-websocket-child-restart.test.ts, the restarts modelled through test seams, with the replacement trying the Home while it has no answer yet: a restart whose drain answers 503 for 5 s ends the interrupted turn on HTTP at 3 retries and on the WebSocket at 5, a 1 s restart ends it on the WebSocket at both, and the next session uses the WebSocket in every case; measured on one Linux x86_64 machine, printed, not asserted); the Home's activity gate keeps its automatic restarts away from busy links (design 5), and a socket Codex loses mid-turn closes the Home leg at once, which the Home records as a hold.

  • OCX_LINK_GATE_CODEX=<absolute path> bun test tests/e2e-style/link-websocket-release-gate.test.ts ×3: exit 0 each, 8 pass, 0 fail, 7 of 7 Codex titles release-labelled (9.3, 9.5, 9.2 s).
  • the same for tests/e2e-style/link-websocket-child-gate.test.ts ×3: exit 0 each, 4 pass, 4 of 4 labelled (5.4, 5.5, 5.6 s).
  • the same for tests/e2e-style/link-websocket-child-restart.test.ts ×3: exit 0 each, 16 pass, 16 of 16 labelled (73.6, 73.8, 74.2 s). The transports and counts are identical in all three runs and to the earlier runs.
  • The three Codex files in one shared bun test process (one module registry): exit 0, 28 pass, 27 labelled. Before this change the second file failed with spawn ENOENT after the first file's cleanup.
  • DOCKER="sudo docker" tests/fixtures/link-ws-autobahn/run.sh <dir> ×3: each from a clean tree with Bun 1.4.4, using crossbario/autobahn-testsuite@sha256:519915fb568b04c9383f70a1c405ae3ff44ab9e35835b085239c258b6fac3074, expectations sha256 ffbc30b651631a6e2f7930894d349348204a88184c252003056d9ae8bd746a26 and spec sha256 16d897ff4f0192f1622f09560fbeb014024785d09168f6a0c85c92c2747fcec3. Each: exit 0; 517 cases, of which 261 pass, 216 unimplemented, 30 closed 1003 and 10 closed 1009; no container left. Report tarball sha256: 8928ea15…8233, 8e5b8691…97ca3, c6de970e…24ee0 (14 s each, on cdab213df), and once more through the changed run.sh on 172432b05: exit 0, the same 517-case classification, tarball 93fd8fb1…85ca, no container left. The framer, the Home leg, the limits and the Autobahn test are unchanged since cea31ad92.
  • run.sh guards: with no argument it prints its usage (exit 1); a run directory that is not empty is refused (exit 2); with DOCKER=true (no server) it stops after 20 s with a message (exit 3).
  • Checks of the new assertions:
    • a fuzzingserver URL on 10.0.0.5 is refused;
    • with a fake codex that exits 3 at start, the restart case, the real-Child Home restart and the gate file's Home restart each fail within a second, naming its stderr;
    • untilOrExit also ends its wait when the run itself rejects before the event, and leaves no unobserved rejection when the run settles after it (probe cases: rejecting run, early exit, late settle);
    • the earlier mutations still hold: the U11 assertion flipped fails; the real Child's limit at 64 MiB fails the header check and then the byte check; the Autobahn binary rule at 1011 gives 30 mismatches.
  • Without the binary or a fuzzingserver, bun test on the five files plus ci-workflows/structure-ssot: exit 0, 57 pass, 29 skip, 0 fail (on 172432b05). tests/test-layout.test.ts, tests/test-layout-tooling.test.ts and tests/ci-workflows/structure-ssot.test.ts: exit 0, 80 pass. bun run structure:check exit 0; bun run privacy:scan exit 0. bun check --no-script --noEmit on the changed test files and helpers: no error in them.
  • Measured for the gate-opening change (no test): the pinned Codex sends no WebSocket ping. During a turn held for 75 s after response.created, a TCP tap saw only its two text messages, and a server ping was answered with a pong within 0.1 s. This matches the source: codex-api/src/endpoint/responses_websocket.rs:138-143 answers pings, and nothing sends one.
  • CI: tests/fixtures/ changes put this PR on the full Ubuntu lane; the full shards must pass on its head.
  • NOT RUN:
    • bun run test and bun run typecheck: repo-wide runs are excluded on this VM, and there is no src/ change;
    • bun run test:changed and hosted CI;
    • the real runtime's supervisor: the restart drain is fired through the tunnel gate's onStopping;
    • the 9.2 controlled validation and U15 (skipped by the owner);
    • the Child RSS acceptance: on its own branch (test(link): the Child's link WebSocket memory acceptance (design 6.3, 11.2-9), opt-in #8686);
    • an Autobahn run against an IPv6 loopback server: the recorded runs use 127.0.0.1;
    • any platform other than Linux x86_64.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed (structure/remote-link.md, release-gate paragraph).
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults (test-only; the gate stays closed; the Autobahn container is published on loopback only).

…nk WebSocket release gate

tests/fixtures/link-gate-codex.json now carries the sha256 of codex-rs built at
openai/codex 88c58562580aa4abe2da6ce4d7676bf171682230 for x86_64-unknown-linux-gnu,
and its note records who built it, when, with which toolchain, target and command.
With this digest, a run of tests/e2e-style/link-websocket-release-gate.test.ts with
that binary is labelled release evidence; any other binary stays a smoke check.
The revision is unchanged. structure/remote-link.md no longer says the fixture
records none, and states that the digest attests one Linux x86_64 build.

Nothing opens the gate: linkWebsocketTransportReady() stays false.
…cket upgrade (U11)

Design T4 U11 needed a release-gate test that Codex's WebSocket client sends no
Origin header; until now the gate showed it only indirectly (the Home refuses an
upgrade carrying one). The relay stand-in now records the Origin header of every
upgrade Codex sends, before forwarding it, and the gate's drain scenario asserts
that every upgrade of both sessions arrived without one.

The release-gate labels, the codex exec runner and the fixture turns move
unchanged into tests/helpers/link-gate-codex.ts, so the real-Child scenarios that
follow label their runs with the same digest check. The gate's scenario bodies
are unchanged; runCodex takes an optional stream_max_retries, default 3 as before.
…ease gate (#8668 review P2-1)

The gate hashed OCX_LINK_GATE_CODEX relative to the working directory and then
spawned the value as given, which Bun looks up on PATH for a bare name. With
OCX_LINK_GATE_CODEX=codex, the attested bytes saved as ./codex and another codex on
PATH, every title said "release gate" while the PATH binary ran; the file could
also be replaced between the hash at module load and the spawns.

The value is now resolved once to an executable regular file (anything else is
refused and the scenarios are skipped), copied read-only into a scratch
directory, and that copy is both hashed and spawned by absolute path
(tests/helpers/link-gate-binary.ts). tests/e2e-style/link-gate-codex-binary.test.ts
covers a bare name with another codex on PATH, a replacement after hashing, and
refused values, with two small scripts instead of Codex; all three cases failed
against the earlier behaviour. structure/remote-link.md describes the copy and
says the smoke rule once, for each Codex scenario's title (P3-1).
…testation note

The note now says the attestation was made in #8668, and lists another build
directory among the things that may change the digest (rustc embeds absolute
paths). Review P3-2.
…-gate-evidence

The pinned-Codex copy (tests/helpers/link-gate-binary.ts) now backs the shared labels in tests/helpers/link-gate-codex.ts, which item 4 moved out of the gate file.
…py up on every path

Review of the previous commit:
- tests/e2e-style/link-gate-codex-binary.test.ts used #!/bin/sh scripts and a
  colon-separated PATH, which the Windows shards cannot run. The stand-ins for
  Codex are now copies of the running Bun executable named codex (codex.exe on
  Windows); each prints its own process.execPath, so a run shows which file
  executed. PATH is joined with path.delimiter, and the execute-permission case
  is POSIX-only. Against the earlier behaviour the bare-name case reports the
  PATH copy (bin/codex) where the hashed file (work/codex) was expected.
- The copy keeps the file's name, so an extension a platform needs stays.
- The gate registers the scratch cleanup before copying, and removes the
  scratch directory at once when the value is refused or preparing it throws.
…nk-gate-evidence

The cleanup-on-error of the pinned-Codex copy moves into tests/helpers/link-gate-codex.ts, which item 4 made the owner of the copy; the fixture is read before the copy is made.
…eal Child relay

tests/e2e-style/link-websocket-child-gate.test.ts drives the pinned Codex through
the real TypeScript Child listener and relay (lane T6) in front of a real Home,
where the gate file uses a test-only stand-in. It is opt-in on
OCX_LINK_GATE_CODEX and labelled exactly as the gate file. Scenarios:
- the switch goes off under a running turn: both requests of the turn run on the
  Child's one socket, and the next session's upgrade is answered 426 by the Child
  (not_advertised) and completes over HTTP;
- the drain past its deadline: the tool result's create is fenced and never
  reaches the upstream, and the same request completes over HTTP;
- design 11.2-8 (a): with the Child's own 101 carrying the 8 MiB limit, a larger
  request goes over HTTP while the socket is open;
- a Home restart mid-turn: the turn completes after Codex reconnects (logged:
  the cut request ends 502 on the socket and Codex completes on a new socket).
It observes the Child's link WebSocket diagnostics and the Home's request-log
rows. Which side closes a fenced socket first is a race between the Home and
Codex exiting, so the close is asserted without its side.
…went on the real Child's socket

The real-Child 11.2-8 (a) scenario asserted only that a socket opened, that no
upgrade fell back and that the Home saw one HTTP row; an oversized message the
Child refused with 1009 before the Home, followed by an HTTP retry, would have
passed it too. A TCP tap between Codex and the Child now records each
connection's request head, the Child's answer head and the bytes Codex sent after
its request head. The scenario asserts that the Child's 101 carries
x-codex-websocket-max-message-bytes at the 8 MiB limit, that Codex wrote less
than the limit on every upgraded connection, and that the whole body went on one
POST connection of its own. Raising the Child's limit to 64 MiB makes Codex send
8,408,470 bytes on the socket and the scenario fail.
… value, not the Child constant

The 11.2-8 (a) scenario took its expected header value and its size threshold from LINK_CHILD_WS_LIMITS.localMessageBytes, the constant the Child itself uses, and matched the header as a substring. It now parses x-codex-websocket-max-message-bytes and requires exactly 8388608 (design T4 6.1), and uses that fixed value for the request size and the byte checks.
…enarios

bun check reported the tap callbacks taking string | Buffer; they receive Buffers. No behaviour change.
… turn, today and with T6's two proposed fixes

tests/e2e-style/link-websocket-child-restart.test.ts restarts the real
TypeScript Child between the two requests of a pinned-Codex turn, in the order
the runtime follows since #8658: the tunnel gate's onStopping drains the relay
(sockets close 1001, upgrades get 503), the old listener stays bound for the
drain phase, stops (refused), and a replacement binds the same port with no
advertisement remembered. Variants model lane T6's proposals before they
exist: the listener closing when the drain starts, and the replacement treating
"no advertisement since start" as "try" (the relay's advertised seam), and both.
Each runs with stream_max_retries 3 and 5, for a 5.5 s and a 1 s restart.

The cases assert only that the turn completes; the transport it ends on, the
503s, the replacement's 426s and Codex's retry notices are printed as one JSON
line per case. The restart phases are wall-clock waits against an external
process, by design. runGateCodex gains an optional stderr line callback.
scripts/test-layout/layout.json: both sides appended to the explicit map; the union keeps link-websocket-traffic.test.ts from #8647 and link-gate-codex-binary.test.ts from this branch. tests/fixtures/test-layout-expected.json auto-merged to the same set.
… pinned binary (#8668 review P3-3)

structure/remote-link.md and the gate file comment said every case is skipped when OCX_LINK_GATE_CODEX is absent; the pin test always runs.
…t from the hashed file (#8668 review P3-4)

Both stand-ins were copies of the same Bun executable, so the digest assertion could not tell work/codex from bin/codex. The PATH-side file now holds other bytes (mode 0755; the gate must never run it), and the test asserts the digest differs from its hash. Against c28e59a behaviour the bare name runs that PATH file (ENOEXEC), a replaced file is what runs (ENOEXEC), and a directory throws EISDIR while hashing; a 0644 file was already refused.
…8668 review P3-5)

The copy now uses COPYFILE_EXCL | COPYFILE_FICLONE, so the 1.6 GB binary costs no space on APFS, btrfs or XFS, and an interrupted direct run leaves nothing large behind there. Checked on Bun 1.4.4: with FICLONE the exclusive flag still refuses an existing destination (EEXIST) and keeps it; on ext4 the copy falls back to an ordinary one. A new case makes the first copy writable and requires EEXIST for a second copy into the same scratch directory; without COPYFILE_EXCL it fails.
…w nits) into test/link-gate-evidence

layout.json: the union of dev, the binary test and the two real-Child files; structure/remote-link.md: the P3-3 wording on top of the U11 and real-Child sentences.
The codex exit mark was written after the replacement Child was bound and the run awaited, so a turn that ended early (over HTTP through the draining Child) had its boundary at the bind. It is now marked when the process exits, and the first-session counts stop there.
…ot after its output is collected

runGateCodex resolves once the process exited and its stdout and stderr were read; the restart measurement marked the exit there, which output draining could delay. runGateCodex gains an onExit callback fired from child.exited, and the measurement uses it to bound the first session.
… Child link WebSocket leg (design 3.3)

tests/clients/link-ws-autobahn.test.ts drives every fuzzingserver case through the Child's own 101 check and a real HomeLeg, with an echo stub in place of Codex that sends one message at a time as Codex does. It is opt-in on a running fuzzingserver (OCX_LINK_WS_AUTOBAHN_URL) and its report directory (OCX_LINK_WS_AUTOBAHN_REPORTS), and it judges each case against reviewed rules in tests/fixtures/link-ws-autobahn-expectations.json: deflate cases unimplemented (the Child offers no extension), a Home message over 4 MiB + 64 KiB or over 4,096 data frames closed 1009, other binary messages closed 1003, everything else OK, NON-STRICT or INFORMATIONAL. Against crossbario/autobahn-testsuite@sha256:519915fb...3074 all 517 cases land in their class.
…dict rests on

Review of 559c34c: a refusal (1003, 1009) now also requires, from the case report, that the Child sent nothing but a close frame and that the close was clean with no handshake or drop timeout; the deflate cases require UNIMPLEMENTED with a clean OK close; each run uses its own agent name, so a report left in the directory by an earlier run cannot supply the verdict, and a control exchange that does not close 1000 fails the run; setting only one of the two variables fails and names the other; the run prints the expectations sha256, the agent and the server.
…ode (1005)

The fuzzingserver ends getCaseCount and updateReports with a close frame that carries no code, which Bun reports as 1005; a dropped connection (1006) or any other code still fails the run.
…rame statistics

A missing or malformed rxFrameStats was read as no frames, which would pass the no-data check; it now fails the case, and a refusal must also show the Child's close frame.
…-gate-evidence

The layout maps (layout.json, layout-clients.json, test-layout-expected.json) are the union of both sides, checked key by key. structure/remote-link.md keeps both sides: this branch's paragraph is dev's plus the U11, real-Child, restart and Autobahn sentences. The gate file keeps this branch's version, in which #8668's labels and binary copy live in tests/helpers/link-gate-codex.ts.
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature). label Nov 13, 2026
…he stated limits against the Child (#8681 review P3-2)

The comment said the clean close is checked for every class, but the pass class did not use it; it now does. The expectations file states the Home message and fragment limits its 1009 rules are written for; the run now asserts they equal LINK_CHILD_WS_LIMITS.
…ve it at exit (#8681 review P3-3)

The copy lives once per module registry. Each file removed it from its afterAll, so a plain multi-file bun test (one shared registry) removed it under the next file, whose scenarios then failed with spawn ENOENT while titled release gate (reproduced: real-Child file then gate file, 7 failures). The helper now removes it once, when the process exits, and at once when the value is refused or preparing it throws; the per-file afterAll calls are gone. The same two files in one run now pass 12 of 12, and a plain process that loaded the helper leaves no copy behind.
#8681 review P3-4)

The titles said the idle socket is fenced and the next create is fenced on the socket, but the assertions accept a close from either side, and Codex closed first in every recorded run. The titles now say what is asserted: the turn finishes on the socket (or its tool result never reaches the upstream over it), the socket closes from either side, and the request completes over HTTP. The fence itself is shown by the stand-in scenarios in the gate file.
…cenario (#8681 review P3-5)

At least one upgraded connection allowed a loop in which Codex ignores the advertised limit, is cut under 8 MiB per attempt and ends on HTTP with every assertion passing. The scenario now requires exactly one upgraded connection through the tap and exactly one link_ws_open record.
…cenarios wait on time (#8681 review P3-6)

The restart measurement waited for the tool result to reach the Home, and the real-Child Home restart for the first request, without racing Codex exiting: a binary that fails early cost the 180 s bound per case, with no stderr. untilOrExit (tests/helpers/link-gate-codex.ts) ends such a wait at once with Codex exit code and the tail of its stderr, and leaves nothing behind when Codex exits after the event. With a fake codex that exits 3 at start, both cases now fail within a second naming it. The helper header no longer says the scenarios never sleep: the restart phases are wall-clock waits by design.
… as measured (#8681 review P3-7)

structure/remote-link.md said the drain 503 lets Codex keep its WebSocket session for the replacement instead of falling back to HTTP. With the pinned Codex every 503, like a refused or dropped connect, spends one stream retry, and the four today cases of the restart measurement ended the interrupted turn on HTTP. Only that clause of the sentence changes.
layout-clients.json and test-layout-expected.json: the union of both sides (link-relay-wait-observer.test.ts from #8680, link-ws-autobahn.test.ts from this branch), checked key by key; layout.json and structure/remote-link.md merged without conflict.
Review of G-2-3: the gate file Home restart scenario still awaited the first request without racing Codex exiting. It now uses untilOrExit, as the helper header says every wait on an event does; with a fake codex that exits 3 at start it fails within a second naming its stderr.
…the event

Review of G-2-3: a rejected run (its output could not be read) was swallowed, so the wait held until the test bound. It now rejects the wait when it comes before the event, and is still handled after it. Checked with three probe cases (rejecting run, early exit, late settle): all pass with no unhandled rejection.
… not by a fence they do not establish

Review of G-2-3: comments in the real-Child file still said the create was fenced; the assertions accept a close from either side and establish only the upstream and transport outcomes. The comments now say so and point to the stand-in scenario for the fence.
… brackets

Review of G-2-3: the loopback check accepts http://[::1]:9001, whose URL.hostname keeps the brackets, which node:net cannot connect to. The case runner now strips them. The recorded runs use 127.0.0.1 and are unaffected.
…the restart measurement; the old rule stays for comparison
@lidge-jun lidge-jun changed the title test(link): release-gate evidence for the link WebSocket: no Origin from Codex, the scenarios through the real Child, a Child restart measurement, Codex's retry facts, and the Autobahn client run test(link): release-gate harness for the link WebSocket: a no-Origin assertion, the gate scenarios through the real Child, a Child restart measurement, and the Autobahn client conformance run Nov 13, 2026
@lidge-jun
lidge-jun marked this pull request as ready for review November 13, 2026 11:30
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner November 13, 2026 11:30
@lidge-jun
lidge-jun merged commit eb727e1 into dev Nov 13, 2026
30 checks passed
@lidge-jun
lidge-jun deleted the test/link-gate-evidence branch November 13, 2026 11:30
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-13T11:32:27.348288Z 20a0bd8 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 20a0bd8128

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

flushing = true;
leg.fromCodex(next);
const done = () => { flushing = false; pump(); };
if (socket.writableLength === 0) queueMicrotask(done); else socket.once("drain", done);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resume the echo queue on write completion, not just drain

When an echoed frame remains buffered but is smaller than the socket's high-water mark, writableLength is positive even though socket.write() returned true. Such a write does not trigger a drain event when it finishes, so this branch leaves flushing true indefinitely and stops echoing subsequent messages. Under socket backpressure, the Autobahn run can therefore report conformance failures caused by the harness rather than the Child leg. Resume the pump from write completion, or otherwise distinguish pending writes from writes that actually require a drain event.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant