Skip to content

fix(relay): report an untyped SSE error envelope as response.failed (re-land #8299) - #8415

Merged
lidge-jun merged 56 commits into
devfrom
reland/pr-8299
Nov 13, 2026
Merged

lidge-jun merged 56 commits into
devfrom
reland/pr-8299

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

Re-land of #8299 (reverted in #8402) with the fixes its post-merge review required.

Some OpenAI-compatible gateways end a stream with an SSE frame that is only {"error":{...}} (no root type), then [DONE] and EOF. Before this change both bare-error recognizers in src/server/relay.ts ignored it, so EOF synthesized response.incomplete / adapter_eof and the diagnostic was lost. After it, the untyped envelope is reported as response.failed across the tee, eager and WebSocket paths, and the request row and account inspector record the same verdict.

What changed relative to the reverted #8299:

  • Selected-credential masking on the untyped policy terminal. An untyped cyber_policy envelope is rewritten into a terminal like the typed error event. The generated terminal now goes through the selected-credential mask on every path, including the WebSocket pump.

  • One precedence rule for bare evidence (src/server/bare-upstream-error.ts): the newest typed error frame wins, an untyped envelope replaces only untyped evidence, and a genuine response.* terminal supersedes both. The client tail, the account inspector, the request row and the WS row all read the same reduced verdict, so a typed quota followed by an untyped unknown no longer reaches the client as a retryable 502 while the row records 429.

  • Status attribution: an untyped rate limit records 429 instead of 502, and a read failure after a captured refusal keeps the refusal verdict for accounting.

  • Explicit codes outrank copy: policy wording no longer overrides an explicit non-policy code.

  • Provider-stated status kept separate from the synthetic 502. The verdict carries httpStatus only when the provider stated one (explicit refusal, recognized class, Codex WS wrapper 5xx), plus a terminalStatus for the wire. A typed error with no recognized class therefore leaves the row open as dev does, so combo preflight still reads explicit status/http_status/numeric code fields, and an ordinary error followed by a socket reset still reports upstream_reset.

  • Policy frames decide only when they win. A per-stream terminal tracker applies the same reducer before the untyped-policy branch, and the client boundary, the inspector, passthrough inspection and the buffered JSON path all use it. An untyped `cyber_policy` envelope that loses to earlier typed evidence is no longer rewritten into a terminal. A winning policy terminal is final and carries the policy verdict, so a typed frame with a recognized code beside policy copy records 400/`cyber_policy` in the row as well as on the client, as dev's row did.

  • Eager reset attribution matches tee and dev: the eager relay marks a read-failure tail as an upstream verdict only for a refusal or provider-stated class, so an unrecognized error followed by a reset keeps the `streamAborted` transport attribution.

  • A bare frame after a genuine terminal leaves the row alone. Readers that keep inspecting past the terminal (the WebSocket EOF observer and the response-log metadata consumer) no longer let a later bare error restart the verdict. Before this, a `max_output_tokens` or completed turn could record 429 and reach account health as a quota failure. The frame's text keeps dev's guard: untyped text that echoes an outbound credential is recorded as `[REDACTED]`.

In buffered non-stream mode, a winning untyped policy frame after a genuine terminal yields `missing_terminal`. That is a deliberate choice: dev already gives `missing_terminal` for the same typed sequence.

  • Upstream copy is masked before it is bounded. A message is cut to 512 characters only after the relay's per-request outbound mask and the outbound credential registry have been applied, so a credential longer than the bound can't reach a client or the request context as a prefix. Both relays now give their mask to the client boundary, and the eager policy rewrite masks before it truncates. A code or type longer than 128 characters is dropped instead of truncated, as dev's diagnostic token check already did.

  • The relay's own tail keeps its verdict on the WebSocket row. The WS log reads the client stream, so it also sees the relay's synthesized response.failed. When that frame's type and code are the winning verdict's own rendering, the row keeps the verdict and doesn't rerun message inference on the provider text it carries. Before this, "unauthorized" or "rate limit" text could turn a 502 into a credential or quota outcome.

  • A raw WebSocket upstream keeps its protocol error. A native-control body has no relay tail, so a typeless envelope there stays the websocket_protocol_error it was on dev, and it isn't recorded as a verdict the client never received.

  • Completed rows drop untyped diagnostics. A genuine terminal that retires untyped evidence clears the code and type that evidence wrote.

  • Credentials in keys and escaped spellings are masked. The outbound mask also replaces a secret's JSON-escaped spelling, and terminal redaction masks object keys as well as values. A rewritten policy terminal copies extra root fields, so a quoted credential used as a key could otherwise survive.

  • A refusal is read from the provider's words. Refusal wording is classified from the bounded, generically redacted provider message before the credential guard runs. A policy or safety refusal whose message also echoes a sent credential therefore stays a refusal for the client, the row and the account, and the stored copy is still guarded.

  • The relay tail must match the verdict's text. The request log treats a response.failed as the relay's own tail only when its type, code and message all equal the verdict's rendering. A genuine upstream terminal with the generic identity is still classified on its own.

  • The buffered path masks before it truncates. The non-streaming collector gets the request's mask, as both relays do. Bare copy is refused ([REDACTED]) whenever the outbound registry can't vouch for every credential it was given, for example one too large to store.

  • The exact credential mask runs before pattern redaction. The terminal redaction walk and both synthetic tail builders mask the selected credential before redactSecrets / redactSecretString run. A pattern rule matching a token inside the credential (sk-…, Bearer …) can therefore no longer split it and leave a prefix the exact mask misses.

  • The final row's error code follows the verdict. When a winning bare verdict names a provider class next to policy-like text (for example rate_limit_exceeded with "flagged for possible cybersecurity risk"), the final row and attempt telemetry now record the verdict's code. Before this, they reclassified the stored text as cyber_policy while the client and the account saw a 429.

  • The relay tail is identified by provenance. Both relays report the exact payload of every bare-verdict tail they synthesize (onSyntheticTail), and passthrough delivery registers it on the request's log context. The request log treats a terminal as the relay's own tail only when its text matches a registered payload, and each registration is consumed once. A genuine upstream terminal, even one with the identical JSON, is always classified as genuine.

Behavior note for reviewers: an untyped cyber_policy envelope ends the turn immediately and drops a later response.completed. That matches the typed error + cyber_policy behavior already on dev (checked with the same two-frame stream against aa30ea9d: typed → response.failed, no completion). It is parity with an existing rule, not a new one.

Combo preflight is unchanged: it treats the untyped frame as a non-control event and commits the target, so it never resends it; the relay reports the failure. A test pins that boundary next to dev's newer bare-error cases.

Closes #8216

Verification

The head merges current `dev` (`24024b4d`) into the re-land chain. Both sides were kept in all three conflicts: two docs/test hunks and the WS test import block. Bun 1.4.4 / macOS, with isolated `HOME`/`OPENCODEX_HOME`/`CODEX_HOME`/`CLAUDE_CONFIG_DIR`.

  • `bun test --isolate` over 52 files: the 12 relay/WS/combo/refusal/reset suites, response-log inspection and request-log suites, the policy-fidelity, stream-aborted, attribution and diagnostics suites, every test that references `buffered-sse-json`, `passthrough-dispatch`, `terminalStatusFromParsed` or `cyber_policy`, and the layout, ratchet, lab-boundary, import-cycle and structure guards. Result: 5,168 pass / 0 fail across 113 files, including the sync suites and every test that references the WS bridge or handler.
  • New regressions in `tests/responses/sse-failed-tail.test.ts`: "a policy frame decides client, account and row together" (5 orderings × tee/eager) and "a reset after … keeps tee and eager attribution aligned" (3 cases). With the source changes reverted, 5 of the 8 fail; with them applied, all pass. Another case, "a bare frame after a genuine terminal leaves the row alone" (5 tests), fails 4 of 5 without its fix. Its redaction test fails if the post-terminal guard is removed. The long-credential cases ("never emits a prefix of a long selected credential" for tee and eager, and "the row keeps no prefix of a long credential this process sent") all fail without their fix. So do the WS tail, raw-body and completed-row cases (4 tests) and the quoted-key case (tee and eager, in `tests/responses/ws-endpoint.test.ts`). The same holds for the refusal-with-credential cases, the generic-identity terminal case, and the buffered long-credential and incomplete-registry cases. It also holds for the pattern-token credential case in `tests/responses/ws-endpoint.test.ts` (tee and eager). After merging `7e3cefd1`, the core relay/WS/log suites and guards pass at 1,041/0. Four of feat(link): the Home accepts the Responses WebSocket over a Remote Link, behind a closed release gate #8593's new link WebSocket tests time out at 60 s on this machine; they time out on `dev` too. The final-row error-code case ("the final row's error code follows the bare verdict") fails without its fix; its copy-only policy control passes either way. The same goes for the WS final-row/attempt case and the genuine-terminal-versus-tail case, which fail on the code before their fixes. The genuine terminal in the relay's exact tail shape also fails against the shape-matching version. The `passthrough-abort` source test now also requires the new `onSyntheticTail` option. After merging `24024b4d`, the core suites and guards pass at 1,037/0.
  • `bun run typecheck` (2,434 files), `bun run privacy:scan`, `bun run structure:check`, `bun scripts/file-size-ratchet.ts` and `git diff --check`: passed.
  • Exact-head CI: see the checks on this head.
  • NOT RUN: full `bun run test`; native Windows/macOS lanes (no per-PR `lane=all`, by policy).

Two test assertions from the precedence commit were corrected rather than loosened. The mixed-order matrix now asserts the status the failed row records (`httpStatusForRequestLogTerminal`), which still rejects a stale losing status. The "reset after an ordinary untyped error" case now asserts the `upstream_reset` that its own title and comment describe, matching dev's typed rule in `tests/server/relay-eager.test.ts`. The mixed-order harness now passes the status argument from `onTerminal` through, as the production recorder does.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. Independent correctness and security reviews run on the final head before merge.

Co-authored-by: Ingwannu ingwannu@users.noreply.github.com

Summary by CodeRabbit

  • Bug Fixes
    • Upstream refusals and recognized provider errors now show more consistent error details and status codes in streaming responses and request logs.
    • Credentials are masked in error messages and diagnostics, including structured data and WebSocket responses.
    • Untyped upstream errors are handled more consistently across streaming and WebSocket delivery, with terminal outcomes reflected in client responses.
    • Genuine terminal events and client cancellations take precedence over provisional errors.
  • Documentation
    • Clarified how untyped error responses are handled during combo streaming.

…8299)

* fix(relay): report an untyped SSE error envelope as response.failed

Some OpenAI-compatible gateways end a stream with an SSE data frame that
is only `{"error":{...}}`, with no root `type`, followed by `[DONE]` and
EOF. Both bare-error recognizers in src/server/relay.ts required a root
`type: "error"`, so the frame was ignored and EOF synthesized
`response.incomplete` with `adapter_eof`, dropping the diagnostic.

Accept a frame with no root `type` that carries an `error` object in
both `boundedBareUpstreamErrorMessage` (tee inspection) and
`boundedBareUpstreamError` (client relay, tee and eager). A typed event
of any other kind is still never treated as a bare error. Redaction,
message bounds, class/code mapping and terminal precedence are unchanged.

The combo preflight is left as is: it treats the untyped frame as a
non-control event and commits the target, so it never resends it; the
relay now reports the failure. A test pins that boundary.

Closes #8216

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(relay): finish untyped error envelope handling across readers

- Treat an untyped {"error":{...}} cyber_policy refusal as the policy
  terminal in both rewrite selection and terminal detection, like the
  typed error event, so later bytes are dropped.
- Tee inspection uses the same structured extraction as the client
  boundary, so a refusal code without a message reports failed.
- A code or error class without a message keeps a bounded fallback
  diagnostic and ends in response.failed instead of adapter_eof.
- The SSE-to-WebSocket pump skips the untyped envelope instead of
  failing it as invalid JSON, so the boundary's terminal reaches WS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9aeffab)
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review
Walkthrough

The change classifies typed errors and untyped error envelopes, reduces evidence across SSE frames, and uses the selected verdict for terminal failures, status reporting, and request logs. Relay and WebSocket paths mask credentials in diagnostic copy.

Changes: Bare upstream error flow

Layer / File(s) Summary
Error classification and credential masking
src/server/bare-upstream-error.ts, src/lib/outbound-credential-registry.ts, src/server/responses/terminal-error-redaction.ts
New helpers classify typed errors and untyped envelopes, derive and reduce bounded verdicts, and mask credential spans in diagnostic copy.
Relay verdicts and terminal output
src/server/relay.ts, src/server/relay-eager.ts, src/server/responses/buffered-sse-json.ts, src/server/responses/passthrough-delivery.ts, src/server/responses/passthrough-dispatch.ts
SSE relays and inspection use the selected verdict for synthetic tails and terminal status. Passthrough delivery supplies credential masking and reports synthesized tails.
Request and attempt attribution
src/server/request-log-terminal-status.ts, src/server/request-log.ts
Request diagnostics track provisional verdicts and relay-generated tails. Request and attempt error-code selection uses the stored verdict when applicable.
WebSocket envelope handling
src/server/index/websocket-handler.ts, src/server/ws-bridge.ts
WebSocket handling distinguishes raw upstream bodies from relayed streams. Relayed untyped envelopes are withheld until the relay emits a terminal event.
Validation and transport contracts
tests/responses/*, tests/routing/combo-stream-preflight.test.ts, tests/usage/upstream-failure-diagnostics.test.ts, scripts/test-layout/layout.json, tests/fixtures/test-layout-expected.json, structure/transports/responses-wire-shapes.md, structure/transports/responses-combo-boundaries.md
Tests cover error recognition, verdict precedence, request logging, credential masking, and WebSocket delivery. Transport documentation describes untyped envelope handling.

Estimated code review effort: 4 (Complex) | ~50 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant UpstreamSSE
  participant SseTerminalOutputBoundary
  participant Relay
  participant RequestLog
  UpstreamSSE->>SseTerminalOutputBoundary: untyped error envelope
  SseTerminalOutputBoundary->>SseTerminalOutputBoundary: reduce verdict evidence
  SseTerminalOutputBoundary->>Relay: winning verdict
  Relay->>RequestLog: synthesized terminal payload and verdict
Loading

Possibly related PRs

Merge Risk: Moderate · previous review 5712b

Upstream error text that echoes a request credential can still reach clients through malformed terminal frames and through untyped policy envelopes with extra fields. Close both before merging.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: an untyped SSE error envelope is now reported as response.failed. The raw summary and objectives confirm this is the core fix, with the untyped envelope handling sprea…
Linked Issues check Passed The only directly linked issue, #8216, is closed. The issue body states: "Closed, completed, canceled, or archived issue. Historical context only; not a requirement for this pull request." No active d…
Out of Scope Changes check Passed The whole-PR summary maps each changed file to the bare upstream error path, its verdict reduction, its row and telemetry recording, or the credential masking that bare-error copy requires. Examples: …
Full details: Title check

Explanation

The title names the main change: an untyped SSE error envelope is now reported as response.failed. The raw summary and objectives confirm this is the core fix, with the untyped envelope handling spread across the relay, request-log, and WebSocket paths. The title also names the relay scope, and the description of the PR matches it.

Full details: Linked Issues check

Explanation

The only directly linked issue, #8216, is closed. The issue body states: "Closed, completed, canceled, or archived issue. Historical context only; not a requirement for this pull request." No active directly linked target remains, so no linked-issue coding requirements apply. For context, #8216 reports that an untyped top-level {"error":{...}} SSE frame, followed by [DONE] and EOF, is reported as response.incomplete with adapter_eof by the native Responses relay. The whole-PR summary shows that the change adds a classifier and reducer in src/server/bare-upstream-error.ts and changes the tee and eager tails in src/server/relay.ts and src/server/relay-eager.ts. The new tests in tests/responses/sse-failed-tail.test.ts cover this envelope. This verdict does not depend on the #8216 requirements. The source lines were not read directly for this verdict.

Full details: Out of Scope Changes check

Explanation

The whole-PR summary maps each changed file to the bare upstream error path, its verdict reduction, its row and telemetry recording, or the credential masking that bare-error copy requires. Examples: src/server/bare-upstream-error.ts (classifier, reducer, guarded copy); src/server/relay.ts and src/server/relay-eager.ts (tails, onSyntheticTail, mask before truncate); src/server/ws-bridge.ts and src/server/index/websocket-handler.ts (WebSocket pump and rawUpstream handling); src/server/request-log-terminal-status.ts and src/server/request-log.ts (row verdict and error code); src/server/responses/buffered-sse-json.ts, passthrough-delivery.ts and passthrough-dispatch.ts (shared tracker and mask wiring); src/server/responses/terminal-error-redaction.ts and src/lib/outbound-credential-registry.ts (masking). The test and layout files support these paths. No unrelated cleanup appears in the summary. The scope is broad, and parts go beyond the minimal #8216 fix, such as key masking, escaped-spelling masking and row error-code selection. Each part still ties to bare-error copy, verdict, or masking, so no demonstrated unrelated change exists. Limits: the diff was not read line by line. Changes since the previous reviewed heads were not separately inspected. This verdict relies on the whole-PR summary and the PR description.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

lidge-jun and others added 2 commits November 9, 2026 10:08
Keep upstream error classification consistent through streaming and
accounting, with bounded client diagnostics and terminal precedence.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
@github-actions github-actions Bot added the bug Something isn't working label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

lidge-jun and others added 5 commits November 9, 2026 13:59
Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Typed and untyped bare error frames could disagree: the client tail
remembered the last bare frame while the request row kept a typed-first
provisional status, so a typed quota followed by an untyped unknown
shipped a generic retryable 502 to the client and a 429 row.

Reduce bare evidence once in bare-upstream-error.ts: the newest typed
frame wins, an untyped frame replaces only untyped evidence, and a
genuine terminal supersedes both. The winning verdict derives identity
and status together, and the tee/eager client boundary, account
inspector, request row and WebSocket row all read it. Explicit refusal
copy is recognized beside an unrelated code; losing frames cannot leave
one reader's code beside another's status or copy.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
The unified bare-error verdict always carried a numeric status, falling back
to 502. Two readers depended on that status being absent for an unrecognized
class:

- the request row wrote the 502 into terminalHttpStatus, so combo preflight
  could no longer read a typed error's explicit status/http_status/code fields
  (429, 400, 410 became 502 and a retry decision changed);
- the synthetic read-failure tail treated every bare verdict as recognized,
  so an ordinary error followed by a socket reset reported the provider's
  generic code instead of upstream_reset.

The verdict now exposes httpStatus only when the provider stated one (explicit
refusal, recognized class, Codex WS wrapper 5xx) and terminalStatus for the
wire. A typed unknown leaves the row open as before; an untyped envelope keeps
its provisional 502. The precedence matrix asserts the status the failed row
records, and the reset case asserts the transport failure its own title
describes.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
@lidge-jun lidge-jun changed the title Re-land #8299 (draft: blocked by review) fix(relay): report an untyped SSE error envelope as response.failed (re-land #8299) Nov 12, 2026
jun and others added 4 commits November 12, 2026 19:09
Final-head review found two divergences from dev:

- An untyped cyber_policy envelope rewrote the client tail and forced the
  account status to 400 even when an earlier typed frame owned the verdict,
  while the row kept the typed status (typed quota -> untyped policy gave
  client cyber_policy, account 400, row 429).
- Eager tagged every bare verdict as upstream_error on a read failure, so an
  unrecognized error followed by a reset lost the streamAborted transport
  attribution the tee path and dev both record.

A per-stream terminal tracker now applies the bare-evidence reducer before the
untyped policy branch, and the client boundary, inspector, passthrough
inspection and buffered JSON all use it. A winning policy terminal is final in
the reducer and carries the policy verdict, so a typed frame with a recognized
code beside policy copy records 400/cyber_policy everywhere, as dev's row did.
Eager states upstream_error only for a refusal or provider-stated class, the
same predicate its tail uses.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Readers that keep inspecting past the terminal (the WebSocket EOF observer
and the response-log metadata consumer) fed a later bare error into the row
reducer after the terminal had retired its evidence. The reducer started a
fresh verdict, so an incomplete max_output_tokens or completed turn followed
by a bare quota frame recorded 429 on the row and reached account health as
quota, while the client had already ended on the genuine terminal.

Once a genuine terminal is recorded, a bare frame no longer touches status,
code or type. Its copy keeps dev's guard: untyped text that fails the
outbound-credential check is recorded as [REDACTED]. Drop the unused
untypedProvisionalStatuses set.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
A bound applied before the outbound mask can keep a prefix of a long selected
credential that the full-value mask no longer recognizes. The bare verdict
sliced message (512), code and type (128) first, and the eager policy rewrite
sliced its message before the block rewrite ran, so a long unpatterned
credential reached eager and WebSocket clients as a prefix and the request
context kept the same prefix.

Bare copy now goes through the relay's per-request mask and the outbound
credential registry before it is bounded; the client boundary receives the
mask from both relays, and the policy rewrite masks before slicing. A code or
type longer than 128 characters is not a class and is dropped rather than
truncated, as dev's diagnostic token check already did.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Dev's request-body facts (#8584) added a line to src/server/request-log.ts,
taking the union to 2000 lines. Fold this branch's untypedSse parameter onto
its parameter line to keep the file at 1999.
lidge-jun and others added 13 commits November 13, 2026 05:42
Final-head review found three WebSocket and row divergences from dev:

- The WS log inspects the client stream, so it also saw the relay's own
  synthesized response.failed and re-ran message inference over the provider
  copy it carries: an unrecognized envelope with "unauthorized" or
  "rate limit" copy became a 401 credential or 429 quota outcome while the
  client and the HTTP row had the transient 502. A response.failed whose
  type and code are the winning bare verdict's own rendering now keeps that
  verdict.
- A native-control WS body is the raw upstream with no relay tail, but the
  pump still held back a typeless envelope for a tail that never came and the
  observer stored its verdict, so the protocol-error turn counted as quota.
  The pump now knows a raw body (rawUpstream) and keeps the envelope as the
  protocol error it was on dev, without recording it.
- A genuine terminal that retires untyped evidence now clears the code and
  type that evidence wrote, so a completed row no longer reports cyber_policy.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Final-head security review found that a selected credential containing a
quote or backslash survived as an object key in a rewritten response.failed:
the policy terminal copies extra root fields, the redaction walk masked
string values only, and the raw-text mask missed the JSON-escaped spelling,
so JSON.parse restored the full secret on tee, eager and WebSocket.

The outbound mask now also replaces the JSON-escaped spelling, and the
redaction walk masks object keys as well as values.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
…il by copy

Final-head review found two divergences:

- A copy-only refusal (policy or safety wording) whose message also echoed
  a credential this process sent was classified after the credential guard
  had replaced the whole message, so the HTTP row, account and buffered
  JSON recorded a generic 502 while the client tail carried the refusal.
  The refusal is now read from the bounded, generically redacted provider
  message before the guard; only the guarded copy is stored or shown.
- The request log recognized the relay's own synthesized response.failed by
  type and code alone, so a genuine upstream terminal carrying the generic
  upstream_error / upstream_server_error identity skipped status inference.
  The tail must now also carry the verdict's own message.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Final-head security review found that the non-streaming buffered collector
built its terminal boundary without the per-request mask. With a selected
credential too large for the outbound registry to keep (it marks itself
incomplete instead), the bare verdict's 512-character slice reached the
JSON client as a prefix of that credential.

The buffered collector now takes the request's mask for both passes, as
both relays already do, and bare copy fails closed to [REDACTED] while the
registry cannot vouch for every credential it was given, the same rule the
request-log diagnostics already apply.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Final-head security review found that a selected credential containing a
pattern token (sk-..., Bearer ...) was split by redactSecrets before the
exact outbound mask ran, so the unpatterned prefix survived in the
rewritten untyped frame, the tee response.failed tail built from it, the
WebSocket client and the WebSocket log row.

The terminal redaction walk now masks the exact credential (values and keys)
first and pattern-redacts the result, and both synthetic tail builders mask
before redactSecretString instead of after.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Final-head review found that an untyped frame naming a recognized class
beside policy-like copy (rate_limit_exceeded + "flagged for possible
cybersecurity risk") reached the client and account as rate_limit_exceeded /
429, while the final row and attempt telemetry re-classified the stored copy
and recorded cyber_policy. The verdict already ruled that the structured
class outranks the copy, so while a winning bare verdict owns the row and
its copy reads like policy, the row takes the verdict's code. Every other
row keeps the ordinary classification.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
… tail

Final-head review found that the request log deleted the bare evidence
before recognizing the relay's own synthesized response.failed, so the
early return kept the provisional 429 status but bareVerdictRowErrorCode
could no longer find the verdict: on the WebSocket path the final row and
the attempt recorded cyber_policy for a rate_limit_exceeded verdict with
policy-like copy. The evidence is now retired only for a genuine terminal.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
… fields

Final-head review found that the request log identified the relay's own
synthesized response.failed only by matching type, code and message against
the bare verdict. A genuine upstream response.failed with those same fields
(a recognized quota class beside policy-like copy) was then kept at the
verdict's 429 while the client boundary rewrote it to cyber_policy and the
account recorded 400.

The tail is now recognized only by the exact shape upstreamErrorFailedPayload
renders: type and response at the root; status, error, an identical
last_error and, for a refusal, retryable:false in response; exactly type,
code and message in the error, all equal to the verdict. A provider terminal
carries its own response fields or lacks the duplicated last_error and is
classified normally.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Final-head review showed that no field or shape comparison can tell the
relay's synthesized response.failed from a genuine upstream terminal: an
upstream can send the exact same JSON, and the request row then kept the
earlier bare verdict (429 rate_limit_exceeded) while the client and account
had the genuine terminal's cyber_policy / 400.

Both relays now report the exact payload of every bare-verdict tail they
synthesize (onSyntheticTail), and passthrough delivery registers it on the
request's log context. The request log treats a terminal as the relay tail
only when its text is a registered payload, consuming each registration
once; upstream inspection never sees registered text, so a genuine terminal
is always classified as genuine.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
t added 2 commits November 13, 2026 11:44
…g them per string

Masking every string of a forwarded untyped envelope against every sent value
kept trading one defect for another: protocol fields renamed by a credential
that equals a field name, a recognized code masked so the tee tail and the
inspector disagreed, seconds of synchronous work on a large frame, and leaks
through keys, extra fields, overlaps and oversized values.

- The terminal boundary reduces an untyped envelope's verdict and withholds the
  frame, so SSE clients receive the terminal it decides, as WebSocket clients
  already did. An explicit untyped policy envelope still becomes the policy
  terminal. Classification reads the evidence it always read.
- The relay registers each verdict tail with its verdict, so the WebSocket row,
  which no longer sees the withheld frame, records the verdict from the tail.
- The exact mask never touches the event line, the root type or Responses field
  names, and merges overlapping credential spans before replacing them.
- The registry records overlapping slices of a value too large to keep whole,
  so bare-copy and row guards still refuse any bounded excerpt of it. The
  per-string registry mask is removed.
…cy terminals

- The client rewrite hands the original of each untyped envelope it masks to the
  tee boundary behind it (createRawEvidenceChannel), so the verdict and policy
  detection read the provider's own frame, as the eager boundary and the
  inspector already did. Delivered bytes still come from the masked copy.
- An untyped policy envelope has no protocol fields of its own, so its policy
  terminal keeps only the guarded message, the default policy type and the
  event sequence number; no upstream extra, response field or error type.
- A sent value too large to keep makes the outbound registry incomplete, so copy
  guards fail closed; sampled slices could not recognize every excerpt.
- maskSecretSpans marks one coverage array per text, each secret only past its
  previous occurrence, so work and memory stay linear in the text length.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


  • Fix comments: Fix CodeRabbit comments on this PR
Prompt to fix all issues
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Inline comments:
Review comments at @src/server/relay.ts:
- Line 387: Update the untyped policy-envelope path identified by
isUntypedBareUpstreamError so policyPayload cannot forward arbitrary root or
response fields: construct the terminal response from required protocol fields
and the guarded policy verdict only. Add tee and eager regression cases
confirming extra envelope fields are withheld.

Review comments at @src/server/responses/terminal-error-redaction.ts:
- Line 95: Update the terminal-error redaction fallback around terminalFrame so
malformed and valid non-target JSON payloads cannot pass through unmasked:
extract and mask the data payload, replace it with replaceSseDataPayload, and
preserve the event line separately. Add regression coverage for both payload
cases.

After applying the fix, consider running `coderabbit review --agent` for local
review. Visit https://docs.coderabbit.ai/cli?utm_source=ghpr
Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 25bafdd and 5712bf8.

Files selected for processing: 12

  • scripts/test-layout/layout.json
  • src/lib/outbound-credential-registry.ts
  • src/server/relay-eager.ts
  • src/server/relay.ts
  • src/server/request-log-terminal-status.ts
  • src/server/responses/passthrough-delivery.ts
  • src/server/responses/terminal-error-redaction.ts
  • structure/transports/responses-wire-shapes.md
  • tests/fixtures/test-layout-expected.json
  • tests/responses/passthrough-abort.test.ts
  • tests/responses/sse-failed-tail.test.ts
  • tests/responses/sse-untyped-credential-mask.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.

Comment thread src/server/relay.ts Resolved
t and others added 20 commits November 13, 2026 12:00
…y per request

- The client rewrite passes untyped envelopes through unchanged (no client
  receives them), so the tee boundary classifies the provider's own frame like
  the eager boundary and the inspector; the payload-keyed raw evidence channel
  is removed. An emitted policy payload closes the boundary as the terminal.
- An untyped policy terminal is a fresh event: response.failed block, without
  the upstream comment, extension or event lines.
- createSynthesizedCopyMask guards copy a boundary synthesizes (verdict tails,
  policy terminals, buffered error presentations) against every value this
  request sent, from its own headers, so a global registry eviction cannot open
  it. Request-correlation identifiers stay readable, as in the registry.
…estructure

A response.failed or response.incomplete frame whose JSON does not parse, or
parses to a non-target payload, had only its extension lines masked after the
protocol-safe rewrite. Its data payload is now masked as text and written back
with replaceSseDataPayload; the event line stays as it is.
… classes public

- createSentValueGuard (per request, from its own headers) recognizes a
  credential-named value whole or by any 8+ character excerpt of its bare
  tokens, and other non-correlation values whole; it guards synthesized copy,
  unstructured terminal payloads and, registered on the log context, the
  request row. Work and memory are bounded and it fails closed past its cap.
- Recognized error classes (refusal, policy, rate-limit, overload, client
  classes) are public vocabulary: neither the client rewrite nor the tail
  identity masks them, so every reader classifies the same frame alike.
- A terminal event line over another JSON payload is masked field by field,
  refusing strings that repeat a sent value; unparsable payloads are masked as
  text under the same guard.
- The boundary emits one [DONE] per turn.
… sent values in place

- Row diagnostics (upstreamErrorCode/Type) refuse a value that repeats what
  this request sent, at capture and at projection; closed vocabulary is public.
- createSentValueMask masks only the characters that repeat a sent value and
  returns the scanned head, so upstream text under a terminal line keeps its
  refusal wording for the tee boundary and nothing past the scan window leaves.
  Non-target terminal strings, non-protocol keys, unparsable payloads and SSE
  extension lines all go through it.
- Synthesized copy leaves public error identities (recognized classes, refusal
  codes, the generic upstream failure) as they are, so buffered presentations
  keep the class the streaming tail reports.
…over = tokens and configured excerpts

- The client rewrite passes typed error frames through like untyped envelopes;
  the terminal boundary redacts each typed error frame it forwards after
  reducing its verdict (createForwardedEvidenceRedaction), so tee and eager
  both classify the provider's own words even when a sent value overlaps the
  refusal wording.
- Credential excerpt windows come from every token between separators
  (whitespace, comma, semicolon, =, quotes, backslash), auth scheme words left
  out, so an opaque value containing = keeps excerpt coverage.
- Configured credential keys and tokens are matched by 8+ character excerpt
  too (containsConfiguredCredentialExcerpt), bounded and failing closed; the
  configured view keeps whole-value matching.
Resolves the relay-eager.ts import conflict by keeping both imports.
…policy evidence

- The boundary redacts a typed policy terminal block like a forwarded frame,
  so its comment and extension lines never carry a sent value.
- Upstream copy drops a numeric field whose digits repeat a sent value (array
  elements are replaced), and an untyped policy terminal keeps its
  sequence_number only when it passes the copy mask.
- Excerpt window generation stops at the cap and counts every window it
  produces, so a long repeated credential fails closed instead of growing.
- When masking removes the policy evidence of a genuine terminal, the record
  carries the cyber_policy code, so the tee boundary reaches the policy
  verdict the eager boundary and the inspector reach.
…s the WS row

- Only protocol event names (response.*, error, done, ping) pass untouched; any
  other event name is provider text and is masked like the rest of the frame.
- A registered relay tail carries the verdict the relay reduced from the
  original frames, so a client-stream reader (the WebSocket row) adopts it
  even when it already recorded evidence from a redacted forwarded copy.
maskExtensionLines split frames on LF only, so a CR-only frame read as one data
line and its event or comment lines skipped masking. Lines now split and rejoin
on CRLF, CR and LF, each separator kept as it was.
…ne terminals

- A held [DONE] block is emitted as the bare sentinel, so comment or field
  lines the provider wrote around it never reach the client.
- Genuine response.failed/incomplete copy (strings, non-protocol keys, numbers)
  is masked for every 8+ character excerpt of a sent credential, without
  cutting output and without touching other header values.
- Masking order is exact credentials, then the pattern rules, then excerpts,
  so an excerpt window cannot split a credential or a Bearer token before the
  rules that recognize it whole.
Each distinct matching window rescanned the whole text, so a large genuine
terminal echoing long credential excerpts stalled the event loop for seconds.
Mark coverage while scanning instead, keeping the work linear.
Restarting indexOf one character after each match rechecked a long
self-overlapping secret at every position, so exact masking of a large
terminal was quadratic. Use a Knuth-Morris-Pratt pass after the native
includes() fast path.
Deduplicate secrets, resume the native search past each match for secrets
that cannot overlap themselves, and charge the KMP scans of self-overlapping
secrets to one budget proportional to the text, masking the whole text once
it runs out.
The multipart rule ran as one regex, so every credential-named part header
on a line without a terminator rescanned that line: quadratic in the input,
reached through bare-error classification of an untyped envelope. Apply the
same grammar in one pass after a part-header regex.

Exact secret masking ran one native search per distinct secret. Find all
secrets in one pass keyed by their first 8 characters (short secrets by
their own length), with confirmation and self-overlap scans charged to a
budget proportional to the text that masks the whole text once exhausted.
A long credential whose 8-character prefix repeated near the end of a frame
rebuilt its border table at every candidate, uncharged. Skip positions with
too little text left, build each table once, and charge it to the budget.
A prefix shared by several absent self-overlapping credentials started one
full scan each and exhausted the budget, blanking an ordinary terminal.
Start the scan only once the whole secret is confirmed at a position.
The exact mask rebuilt its secret set for every string of a frame, so a
terminal with many fields cost fields times credentials. Build a SecretIndex
once per request, walk each prefix group shortest first and stop at the first
secret that cannot fit, and keep each call's budget proportional to its own
text so the frame as a whole stays linear.
credentialSetMatches ran one includes() per recorded value over the text,
so classifying a large untyped envelope cost message length times value
count. The registry sets and the configured snapshot now keep a
SecretIndex of their containment-eligible values and check it in one
charged pass that fails closed.
Charging each candidate its full length before comparing meant a few keys
sharing a prefix and a response that mentions the prefix repeatedly
exhausted the budget and blanked ordinary output. Charge the matched length
plus one, which still exhausts the budget on near-copies at every position.
Each segment class excludes the dot, so a segment always runs to the end of
its token run. As a regex, every eyJ inside one long run with no dot after
it rescanned the rest of the run, which untyped-envelope classification
reached with a large upstream message. Skip past a failed run instead; a
400,000-case differential against the regex found no output difference.
@lidge-jun

Copy link
Copy Markdown
Owner Author

Maintainer judgment (final head)

Reviewed head: bde51c426faea8b3b11494d30d0876922ec6c466. This replaces the judgment for 734ca8ff. Since that head the branch merged dev at f4386e79 (an import-only conflict in relay-eager.ts) and took the redaction and bounded-work fixes listed below.

  • Value: unchanged from the earlier judgment. An untyped {"error":{...}} frame from an OpenAI-compatible gateway now ends the turn as response.failed with the provider's verdict, where dev showed response.incomplete / adapter_eof. The tee, eager, buffered JSON and WebSocket readers, the account-health outcome, the request row and the attempt telemetry all read one reduced verdict.
  • UX change: users of those gateways see the real failure (quota, overload, refusal, or a generic upstream error) instead of an EOF. Clients never receive the untyped envelope itself. They get the synthesized terminal, whose copy is guarded against every value the request sent upstream. Genuine response.failed / response.incomplete frames keep their full output and lose only exact credentials, pattern-recognized secrets, and 8+ character excerpts of a sent credential.
  • Regression risk for existing users: moderate surface, now with bounded work. Since 734ca8ff, review rounds found ways a hostile or very large upstream frame could stall the event loop, and each is fixed with a test that fails on the head before its fix:
    • Excerpt masking, exact masking (overlapping, duplicate, thousands of distinct or same-prefix credentials, many fields per frame), and registry and configured-credential matching now each make one pass over the text. Each pass is charged to a budget proportional to the text, and fails closed by masking the whole string.
    • Two redactSecretString rules that already exist on dev were quadratic: multipart and JWT. Both now run in one pass, which also helps every other caller. Random differentials against the previous implementations found 0 differences: 200,000 multipart cases, 400,000 JWT cases, 100,000 masking cases and 100,000 containment cases.
    • The fail-closed budget masks a whole string only when a frame repeats near-copies of a credential at most positions. The reviewers found no ordinary-traffic trigger.
    • Locally on the final head, 28 focused files, including every test that imports redact.ts, gave 1,761 pass / 0 fail. Typecheck, privacy scan, structure check and diff check all pass.
  • Security: this is the fix for the reverted fix(relay): report an untyped SSE error envelope as response.failed #8299's credential exposure. Request-correlation identifiers are out of scope by owner decision. Accepted limits: a sent value under 8 characters inside longer copy is not recognized. A sent value over 16 KB, or a credential set over 65,536 excerpt windows, makes the guards fail closed. The separate final security review on this head passed.
  • Windows: no platform-specific code. Relay, redaction and logging logic is shared. Native Windows and macOS lanes were not run per PR, by policy.

Final-head reviews (gpt-6.1-terra, high): correctness PASS and separate security PASS, both on bde51c42.

The correctness reviewer left two notes, not findings:

  • The wall-clock regression bounds depend on host load; a deterministic work counter would be stronger.
  • Already-scanned secrets in a prefix group are skipped without charge. A 3.7 MB probe with 500 such secrets ran in about 0.1 s.

Follow-ups after merge:

  • Align the older refusal-precedence sentence in structure/transports/responses-wire-shapes.md.
  • Bound the length of the policy error.type.
  • Consider work-count guards for the masking regressions.

@lidge-jun

Copy link
Copy Markdown
Owner Author

Maintainer judgment (merge check)

This adds the exact-head CI result and the latest-dev check to the final-head judgment. Both final reviews passed on bde51c426faea8b3b11494d30d0876922ec6c466: the correctness review and the separate security review, both gpt-6.1-terra at high reasoning effort.

  • CI: finished on bde51c42 with 20 pass, 13 skipping, and nothing failed or pending.
  • Latest dev (de85cf23, 20 commits ahead of the merge base): a local trial merge, never committed or pushed, had no conflicts. Eight files overlap, including request-log.ts, ws-bridge.ts and the test-layout maps.
  • Merged tree checks:
    • bun run typecheck passes (2,496 files) and bun run structure:check passes.
    • 13 isolated test files give 1,431 pass / 0 fail. They cover the overlapping files, the test-layout guards and the file-size ratchet.

Value, UX change, regression risk and Windows impact are as stated in the final-head judgment. Merging with --squash --match-head-commit bde51c426faea8b3b11494d30d0876922ec6c466.

@lidge-jun
lidge-jun merged commit 6d19642 into dev Nov 13, 2026
33 checks passed
@lidge-jun
lidge-jun deleted the reland/pr-8299 branch November 13, 2026 05:57
lidge-jun added a commit that referenced this pull request Nov 13, 2026
… its WsData, as every upgraded socket does

#8415 added a test that hands sendResponseToWebSocket a socket with no data. Every socket the shared Responses server upgrades carries a WsData object (the Responses socket, the live sideband, the Remote Workspace agent and the link gate all pass data to server.upgrade), and the bridge already reads ws.data on its other paths, so the socket in the test is what was wrong; since the Home counts link socket traffic, sendTextFrame reads ws.data.link on every send and the test threw.
lidge-jun added a commit that referenced this pull request Nov 13, 2026
…ance, the Home counts link sockets, and the review nits of #8622 and #8625 (#8647)

* test(responses): pin the combo replay on a paced source, and only the read slots on an unpaced one

* fix(responses): a combo child's required-tool check keeps whether the outbound body denies every tool, not the parsed body

* fix(link): end every relayed request's traffic flow through one guarded helper, so a throwing end never keeps the lease

* fix(link): a throwing stall observer never breaks a relayed body

* test(link): a parsed create stays with its own request when a fallback or recovery builds the next one

* docs(link): note that a link create's parsed body shares its nested input with the frame

* feat(link): the Home counts each link WebSocket in its per-link traffic book with the Child's definition

* test(link): the RSS acceptance announces a stall only after a quiet window, and fails a load that does not settle

* docs(link): only the settled acceptance runs count, and how the quiet window settles them (#8600)

* docs(link): the Home counts link sockets in the Child's units, and says where completeness differs

* docs(link): the traffic counters' comments cover a link socket's message payloads

* docs(responses): the required-tool check keeps its tool declarations as well as whether the body denies every tool

* test(link): a refused or failed link upgrade counts nothing in the Home's traffic book

* style(link): drop the trailing blank line from the gate and the RSS fixture

* docs(link): the quiet window's timers are the only ones that change adds to the RSS fixture

* test(link): the failed-upgrade case proves the gate reached server.upgrade

* test(responses): the workflow ceiling test's hand-made socket carries its WsData, as every upgraded socket does

#8415 added a test that hands sendResponseToWebSocket a socket with no data. Every socket the shared Responses server upgrades carries a WsData object (the Responses socket, the live sideband, the Remote Workspace agent and the link gate all pass data to server.upgrade), and the bridge already reads ws.data on its other paths, so the socket in the test is what was wrong; since the Home counts link socket traffic, sendTextFrame reads ws.data.link on every send and the test threw.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Untyped SSE error envelope is reported as adapter_eof by native Responses relay

2 participants