Repository navigation
fix(link): cap a late grant at the first deadline and keep the orphan flag for an unreadable pidfile - #8610
Merged
Conversation
… flag for an unreadable pidfile A grant line seen after spawn + READY_PROOF_MS, before the next tick stopped the candidate, gave it about 61 s instead of the stated 60 s; the grant now counts from the first deadline at most. A one-route link flagged orphan-unverified that gains routes lifted the flag when its pidfile could not be read or parsed; only a missing pidfile lifts it now. The route-rounds test advances straight to the known retry time instead of looping until a probe appears. Refs #8609
… the next tick stops the candidate Refs #8609
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Contributor
|
✅ Deterministic PR hygiene checks passed. |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
3 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-ups to the multi-route Remote Link work (#8602, #8603) from the final review of #8603. Closes #8609.
A late grant gave a route candidate about 61 s instead of the stated 60 s. A candidate gets
READY_PROOF_MS(30 s) from spawn to see OpenSSH's grant line, then 30 s from the grant for its readiness proof. The grant time was taken as-is, so a grant line printed just after spawn + 30 s, before the next 1 s supervisor tick had stopped the candidate asgrant_unobserved, moved the deadline to grant + 30 s.readinessDeadlineinsrc/link/route-rounds.tsnow counts a grant frommin(grantedAt, spawnedAt + READY_PROOF_MS). A late grant still gets its proof attempts until spawn + 60 s, and the readiness exec timeouts use the same deadline. An in-time grant gets the same deadline as before.An unreadable pidfile lifted the orphan flag. When a one-route link flagged
orphan-unverifiedat startup gains routes, the supervisor either watches the earlier ssh from its pidfile or lifts the flag when no pidfile is left. The default reader returned null on any read or parse error, so a pidfile that existed but could not be read counted as gone, and a candidate could start while the earlier ssh might still hold the port. The cost was bounded (ExitOnForwardFailure=yesmakes the candidate fail, then the route rests 60 s), but it broke the rule that no candidate starts while an earlier tunnel may live.src/link/supervisor.tsnow has a strict loader (nullonly on ENOENT; it throws on any other read error or on a file that does not parse) and a lenient wrapper. Only the flag-lift site uses the strict loader: a throw keeps the flag and logs[link] pidfile unreadable; link stays blocked link=<id>. Every other pidfile read uses the wrapper and behaves as before. One test-only difference: an injecteddeps.readPidfilethat throws used to propagate from every call site, and now does so only at the lift site; the wrapper swallows it elsewhere, as the default reader always did. No current injection throws. The link then shows asfailed/stale tunnel may hold the port, and the way out is the one already documented for an unverifiable pidfile: confirm no ssh for the link is running, delete the pidfile and restart the Home.The "60 s" in the code comments and in
structure/remote-link.mdnow describes the readiness deadline: at most 60 s after spawn. The 1 s supervisor tick after that deadline stops the candidate (SIGTERM, then SIGKILL after 2 s), as before.Smaller fixes. The doc comment on
confirmedLineTrust(src/link/route-trust.ts) said the file has one line for the alias. The code has accepted several lines carrying the same key type and key since #8603, and the comment now says so. The route-supervisor test that looped the fake clock until a probe appeared ("within 61 s") now advances to the known retry time: the forward retry is due at 60 s (HOME_TUNNEL_RETRY_POLICY), its round runs the ssh client check on that tick, and the first probe goes on the next tick.structure/remote-link.mdis updated for the deadline and the unreadable-pidfile case.What keeps this inert. Both behaviour changes apply only to multi-route links (a link record with
routes), which exist only after an operator adds a route withocx link routes add. One-route links take the old path: they run no route rounds, and the strict loader runs only when a link gains routes. No config key, default or one-route code path changes.Out of scope, as decided in #8609: the dashboard's generic message for the five enrollment codes (another lane owns the dashboard), and timing a real slow route against the readiness budget (no such route can be reached from the build machine).
Verification
Tests written first, on
devat e85a975, intests/clients/link-route-supervisor.test.ts:a grant seen after the first deadline, before the tick that would stop the candidate, keeps the candidate within 2 x READY_PROOF_MS: skips READY_PROOF_MS + 500 without a tick, grants, advances READY_PROOF_MS - 501 (no signal), then 1 ms more and expects SIGTERM at exactly 2 x READY_PROOF_MS and areadinessoutcome. Ondevit failed at the SIGTERM assertion (signals[]; the deadline was 2 x READY_PROOF_MS + 500).a flagged one-route link that gains routes while its pidfile …(three cases, using the supervisor's real reader on a temporary pidfile directory). In theis a directoryandholds no pidfilecases,devlifted the flag (orphanundefined), so the cases failed; now the flag stays, no probe runs at the retry time, and nothing is signalled. Theis gonecase (ENOENT) lifts the flag and starts the round on both versions; it pins the default reader's ENOENT contract.… orphan flag left by a failed spawn …test now asserts no ssh run at 59.999 s, the client check at 60 s, and the probe on the next tick.Commands (exit codes). The full focused run below was on 19b6f24; the last commit, 389c9c0, changes only comments and
structure/remote-link.md, and the first five commands plus the guard files were run again on it.TMPDIR=/tmp bun test tests/clients/link-route-supervisor.test.ts: 50 pass, 0 fail (exit 0).bun run typecheck: exit 0.bun run privacy:scan: exit 0.bun run structure:check: exit 0.TMPDIR=/tmp bun scripts/test.ts --parallel=2 tests/clients/link-route-supervisor.test.ts tests/clients/link-boundary.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/structure-ssot.test.ts tests/ci-workflows/ci-structure-gate.test.ts tests/ci-workflows/file-size-ratchet.test.ts: 145 pass, 0 fail (exit 0).TMPDIR=/tmp bun scripts/test.ts --parallel=2 tests/clients tests/server/link-management-routes.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/structure-ssot.test.ts tests/ci-workflows/ci-structure-gate.test.ts tests/ci-workflows/file-size-ratchet.test.ts(204 files, includingtests/clients/link-boundary.test.ts): 3455 pass, 67 skip, 3 fail (exit 1). The 3 failures are the Rust ocx-child tests (tests/clients/link-ocx-child-e2e.test.ts, two cases, and the Rust case oftests/clients/link-ocx-child-conformance.test.ts). They stop before any test code runs, becausecargoon the build machine has no default rustup toolchain. This branch touches no Rust code.NOT RUN: the full
bun run testsuite (the build machine allows focused files only; CI covers the rest); the three Rust ocx-child tests above (no toolchain); Windows and macOS (the changed code is platform-independent; the strict pidfile loader uses the samereadFileSyncas before); a real slow route timed against the readiness budget.Review: two read-only rounds by an independent reviewer (gpt-6.1-terra, xhigh effort). Round 1 on 19b6f24 found one P2: the docs said a candidate never lives past 60 s, while the cap applies to the readiness deadline and the stop comes on the next tick. That is fixed in 389c9c0. Round 2 on 389c9c0 found no P0–P2, ran
tests/clients/link-route-supervisor.test.tson that head (50 pass, 0 fail), and judged the injected-reader difference above below P2. Round 1 confirmed that an in-time grant keeps its previous deadline and classification, that a late grant is classifiedreadiness, that the new tests use only the injected clock and timer, and that the warning carries the link id with no path or host.Checklist
Implemented by Claude Opus 5.5 (anthropic/claude-opus-5-5) in GOAT through the Codex harness; review by GPT-6.1 Terra (xhigh). Received as a git bundle from the VM that built it and pushed by the coordinator.