Skip to content

fix(link): cap a late grant at the first deadline and keep the orphan flag for an unreadable pidfile - #8610

Merged
lidge-jun merged 3 commits into
devfrom
fix/link-routes-followups
Nov 13, 2026
Merged

lidge-jun merged 3 commits into
devfrom
fix/link-routes-followups

Conversation

@lidge-jun

Copy link
Copy Markdown
Owner

Summary

Follow-ups to the multi-route Remote Link work (#8602, #8603) from the final review of #8603. Closes #8609.

A late grant gave a route candidate about 61 s instead of the stated 60 s. A candidate gets READY_PROOF_MS (30 s) from spawn to see OpenSSH's grant line, then 30 s from the grant for its readiness proof. The grant time was taken as-is, so a grant line printed just after spawn + 30 s, before the next 1 s supervisor tick had stopped the candidate as grant_unobserved, moved the deadline to grant + 30 s. readinessDeadline in src/link/route-rounds.ts now counts a grant from min(grantedAt, spawnedAt + READY_PROOF_MS). A late grant still gets its proof attempts until spawn + 60 s, and the readiness exec timeouts use the same deadline. An in-time grant gets the same deadline as before.

An unreadable pidfile lifted the orphan flag. When a one-route link flagged orphan-unverified at startup gains routes, the supervisor either watches the earlier ssh from its pidfile or lifts the flag when no pidfile is left. The default reader returned null on any read or parse error, so a pidfile that existed but could not be read counted as gone, and a candidate could start while the earlier ssh might still hold the port. The cost was bounded (ExitOnForwardFailure=yes makes the candidate fail, then the route rests 60 s), but it broke the rule that no candidate starts while an earlier tunnel may live. src/link/supervisor.ts now has a strict loader (null only on ENOENT; it throws on any other read error or on a file that does not parse) and a lenient wrapper. Only the flag-lift site uses the strict loader: a throw keeps the flag and logs [link] pidfile unreadable; link stays blocked link=<id>. Every other pidfile read uses the wrapper and behaves as before. One test-only difference: an injected deps.readPidfile that throws used to propagate from every call site, and now does so only at the lift site; the wrapper swallows it elsewhere, as the default reader always did. No current injection throws. The link then shows as failed / stale tunnel may hold the port, and the way out is the one already documented for an unverifiable pidfile: confirm no ssh for the link is running, delete the pidfile and restart the Home.

The "60 s" in the code comments and in structure/remote-link.md now describes the readiness deadline: at most 60 s after spawn. The 1 s supervisor tick after that deadline stops the candidate (SIGTERM, then SIGKILL after 2 s), as before.

Smaller fixes. The doc comment on confirmedLineTrust (src/link/route-trust.ts) said the file has one line for the alias. The code has accepted several lines carrying the same key type and key since #8603, and the comment now says so. The route-supervisor test that looped the fake clock until a probe appeared ("within 61 s") now advances to the known retry time: the forward retry is due at 60 s (HOME_TUNNEL_RETRY_POLICY), its round runs the ssh client check on that tick, and the first probe goes on the next tick.

structure/remote-link.md is updated for the deadline and the unreadable-pidfile case.

What keeps this inert. Both behaviour changes apply only to multi-route links (a link record with routes), which exist only after an operator adds a route with ocx link routes add. One-route links take the old path: they run no route rounds, and the strict loader runs only when a link gains routes. No config key, default or one-route code path changes.

Out of scope, as decided in #8609: the dashboard's generic message for the five enrollment codes (another lane owns the dashboard), and timing a real slow route against the readiness budget (no such route can be reached from the build machine).

Verification

Tests written first, on dev at e85a975, in tests/clients/link-route-supervisor.test.ts:

  • a grant seen after the first deadline, before the tick that would stop the candidate, keeps the candidate within 2 x READY_PROOF_MS: skips READY_PROOF_MS + 500 without a tick, grants, advances READY_PROOF_MS - 501 (no signal), then 1 ms more and expects SIGTERM at exactly 2 x READY_PROOF_MS and a readiness outcome. On dev it failed at the SIGTERM assertion (signals []; the deadline was 2 x READY_PROOF_MS + 500).
  • a flagged one-route link that gains routes while its pidfile … (three cases, using the supervisor's real reader on a temporary pidfile directory). In the is a directory and holds no pidfile cases, dev lifted the flag (orphan undefined), so the cases failed; now the flag stays, no probe runs at the retry time, and nothing is signalled. The is gone case (ENOENT) lifts the flag and starts the round on both versions; it pins the default reader's ENOENT contract.
  • The existing … orphan flag left by a failed spawn … test now asserts no ssh run at 59.999 s, the client check at 60 s, and the probe on the next tick.

Commands (exit codes). The full focused run below was on 19b6f24; the last commit, 389c9c0, changes only comments and structure/remote-link.md, and the first five commands plus the guard files were run again on it.

  • TMPDIR=/tmp bun test tests/clients/link-route-supervisor.test.ts: 50 pass, 0 fail (exit 0).
  • bun run typecheck: exit 0.
  • bun run privacy:scan: exit 0.
  • bun run structure:check: exit 0.
  • On 389c9c0: TMPDIR=/tmp bun scripts/test.ts --parallel=2 tests/clients/link-route-supervisor.test.ts tests/clients/link-boundary.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/structure-ssot.test.ts tests/ci-workflows/ci-structure-gate.test.ts tests/ci-workflows/file-size-ratchet.test.ts: 145 pass, 0 fail (exit 0).
  • On 19b6f24: TMPDIR=/tmp bun scripts/test.ts --parallel=2 tests/clients tests/server/link-management-routes.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/structure-ssot.test.ts tests/ci-workflows/ci-structure-gate.test.ts tests/ci-workflows/file-size-ratchet.test.ts (204 files, including tests/clients/link-boundary.test.ts): 3455 pass, 67 skip, 3 fail (exit 1). The 3 failures are the Rust ocx-child tests (tests/clients/link-ocx-child-e2e.test.ts, two cases, and the Rust case of tests/clients/link-ocx-child-conformance.test.ts). They stop before any test code runs, because cargo on the build machine has no default rustup toolchain. This branch touches no Rust code.

NOT RUN: the full bun run test suite (the build machine allows focused files only; CI covers the rest); the three Rust ocx-child tests above (no toolchain); Windows and macOS (the changed code is platform-independent; the strict pidfile loader uses the same readFileSync as before); a real slow route timed against the readiness budget.

Review: two read-only rounds by an independent reviewer (gpt-6.1-terra, xhigh effort). Round 1 on 19b6f24 found one P2: the docs said a candidate never lives past 60 s, while the cap applies to the readiness deadline and the stop comes on the next tick. That is fixed in 389c9c0. Round 2 on 389c9c0 found no P0–P2, ran tests/clients/link-route-supervisor.test.ts on that head (50 pass, 0 fail), and judged the injected-reader difference above below P2. Round 1 confirmed that an in-time grant keeps its previous deadline and classification, that a late grant is classified readiness, that the new tests use only the injected clock and timer, and that the warning carries the link id with no path or host.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Implemented by Claude Opus 5.5 (anthropic/claude-opus-5-5) in GOAT through the Codex harness; review by GPT-6.1 Terra (xhigh). Received as a git bundle from the VM that built it and pushed by the coordinator.

… flag for an unreadable pidfile

A grant line seen after spawn + READY_PROOF_MS, before the next tick stopped the
candidate, gave it about 61 s instead of the stated 60 s; the grant now counts
from the first deadline at most. A one-route link flagged orphan-unverified that
gains routes lifted the flag when its pidfile could not be read or parsed; only a
missing pidfile lifts it now. The route-rounds test advances straight to the known
retry time instead of looping until a probe appears.

Refs #8609
@lidge-jun lidge-jun added the bug Something isn't working label Nov 13, 2026
@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@lidge-jun
lidge-jun marked this pull request as ready for review November 13, 2026 01:16
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner November 13, 2026 01:16
@lidge-jun
lidge-jun merged commit 36ee226 into dev Nov 13, 2026
34 of 36 checks passed
@lidge-jun
lidge-jun deleted the fix/link-routes-followups branch November 13, 2026 01:16
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-13T01:17:38.005111Z 389c9c0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Link routes: cap a late grant at the first deadline, and keep the orphan flag when the pidfile is unreadable

1 participant