Repository navigation
feat(link): let a Remote Link use several SSH routes, preferring direct ones over the relay - #8602
Conversation
…a confirmed-termination ssh run A link record may list several SSH routes to the same machine. The alias stays the pinned host-key name and is required in the list; a one-route link keeps the exact record an older Home reads. Tunnel and exec argv can connect to a route while HostKeyAlias stays the alias, and buildRouteProbeArgv checks a route strictly against the pinned key. SshRunner.run gains an opt-in AbortSignal with one termination path for timeout and abort.
…rrelated readiness proof A link with several routes is connected by selection rounds: each eligible route is probed against the pinned host key, the -R tunnel starts on the first that answers, OpenSSH must confirm the granted remote listener, and the Child must fetch its own catalog with a fresh proof id that this Home admits under the tunnel generation. Failed routes cool per class at today's per-alias rates; the link reports the most recoverable outcome over all routes. One-route links keep the existing path.
…ssion A key-admitted catalog request on the hub-link listener reports the proof id it carried; the optional listener set owns the proof registry and checks it against the tunnel generation the supervisor reports.
GET /api/link/status?routes=1 adds routes, route (name, index, since) and routeBlocked to links with several routes; the default document is unchanged. ocx link status asks for it and validates the fields strictly. An unsupported ssh client reads as a failed multi-route link.
…enSSH ssh -G shows the pinned alias, strict checking and the link known_hosts win over a hostile per-destination config. An opt-in fixture (OCX_OPENSSH_ROUTE_FIXTURE=1) runs a private sshd -i through ProxyCommand: the pinned probe passes, another key is refused without touching known_hosts, a granted -R carries bytes, and a second session or a server without forwarding is refused with OpenSSH's failure line.
… client re-checks for multi-route links
…, and take the proof header off the request A tunnel is owned and watched before its pidfile is written, and a failed write stops it before the next route. An orphan whose identity cannot be read keeps the link blocked and is never signalled; a link whose orphan is gone starts its first round. New probes and tunnels wait while lifecycle work runs, and changed links lose their round before the first stop is awaited. A proof that settles after its candidate exited cannot promote it. The hub-link catalog admission deletes the proof header before dispatch.
…thout a deadline, and recheck proof deadlines A multi-route link blocked by an earlier tunnel keeps a non-signalling watch and starts its first round once the process is gone or its pid names another process. A probe that settles while lifecycle work runs moves to a deferred phase no deadline applies to. A readiness proof promotes only before both its attempt and candidate deadlines.
…te probe results An orphan watch belongs to one multi-route record instance and pidfile; removal, replacement or a change to one route retires it before any further signal, and a pidfile is removed only when it is still exactly the watched one. A route probe that settles after its deadline counts as a timeout even before the tick runs.
…te probe answer as a timeout An orphan watch signals, lifts the block or starts a round only while the pidfile is still exactly the one it watches; a replacement is only watched and keeps the link blocked. A route probe that answers after its deadline is a transient timeout whatever its exit code or stderr says.
…laced An orphan watch keeps every process its pidfile has named; after a replacement nothing is signalled, and the link stays blocked until all of them are gone.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
✅ Deterministic PR hygiene checks passed. |
# Conflicts: # src/cli/link.ts # src/link/status-projection.ts # tests/cli/cli-link-check.test.ts # tests/cli/cli-link.test.ts # tests/server/local-control-socket.test.ts
…the readiness, return and orphan rules
# Conflicts: # src/cli/link.ts # src/link/status-projection.ts # src/server/management/link-routes.ts # tests/cli/cli-link-check.test.ts # tests/cli/cli-link.test.ts # tests/server/local-control-socket.test.ts
… blocks decodes with all three
…ve earlier tunnel
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01f954df79
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| * Ordered SSH routes to the same machine, direct routes first and the relay alias last. | ||
| * Absent means exactly [alias]; a list always contains alias, which stays the pinned host-key | ||
| * name (HostKeyAlias) and the destination of every exec operation. Hub-initiated links only. | ||
| */ | ||
| routes?: string[]; |
There was a problem hiding this comment.
Document multi-route configuration in the public Remote Link guide
The new routes field is user-facing configuration, and the structure document says that manually editing the record is currently the only way to enable it, but this commit makes no corresponding docs-site/ update. Users therefore lack public instructions for configuring the feature, including the required alias entry, the lack of automatic failback, and the fact that older releases reject the modified store on downgrade. Add these instructions and compatibility warnings to the public Remote Link guide, as required by src/AGENTS.md.
AGENTS.md reference: src/AGENTS.md:L29-L29
Useful? React with 👍 / 👎.
| if (routesEdited.some(record => rounds.isMultiRoute(record))) rounds.recheckClient(); | ||
| for (const record of routesEdited) if (rounds.isMultiRoute(record)) rounds.routesChanged(record); |
There was a problem hiding this comment.
Reconsider terminal host-key failure when adding a new route
If every existing route has failed host-key verification, adding a new, working route and reloading never tries it. This branch updates the route books but preserves the reducer's terminal failed(hostkey) state, which has no retryAt; consequently, spawnFor rejects the reload attempt and every subsequent tick, even though the newly added route has never refused the pinned key. A focused reproduction remained terminal after adding a working direct route and advancing the clock by an hour. Recompute or reset the terminal state when a route-list edit introduces an untried route, while preserving any existing live tunnel.
Useful? React with 👍 / 👎.
Summary
A Home-initiated Remote Link names one SSH alias, so when the only alias a Home uses for a Child is a relay alias, every model request of that Child crosses the relay even when a direct path (same LAN, a tailnet address) exists. Re-linking on a direct alias by hand works, but then the relay is gone as a fallback when the direct path is down.
This adds an ordered route list to a link and a selection loop in the Home's tunnel supervisor. Nothing in the product writes the list yet (the management API and
ocx link routescommands come in a follow-up); a link without the field behaves exactly as before.LinkRecord.routes?: string[](src/link/store.ts), 2 to 8 SSH aliases, direct first, relay last. It must containalias, which stays the pinned host-key name and the destination of every exec operation.[alias]is normalized away, so a one-route link keeps the exact record an older Home reads. An older Home rejects a store that lists routes (strictRECORD_KEYS), the same downgrade caveat asremotePlatformandchildKind.HostKeyAliasstays the alias, withStrictHostKeyChecking=yesagainst the link known_hosts file. A route that offers another key fails with "Host key verification failed"; nothing is trusted on first use or written.src/link/route-rounds.ts,src/link/route-select.ts). Routes are probed in order (10 s each, on the supervisor clock). The-Rtunnel starts on the first that answers. OpenSSH's own debug1 line must confirm that the Child granted the remote listener; the stderr tap otherwise keeps exactly today's classifier lines. The tunnel is then proven end to end: the Child fetches its own/v1/catalogcarrying a fresh proof id, its link relay forwards it with relay authentication, and the Home (src/link/route-proof.ts, hooked into the hub-link admission) accepts the id only for that link key, before its deadline, under the current tunnel generation, while the same request returns a catalog. Only then is the link connected (30 s per candidate). The readiness step reads the Child'socx config show --jsonwith the runner's 4 MiB cap, the same limit enrollment uses, so a Child with a large config ([Bug] Remote Link: Child preflight fails whenocx config show --jsonexceeds the 64 KiB SSH output cap #8356) works with several routes too.ocx link reconnectstarts a new round from the first rested route.failedwith reasonstale tunnel may hold the port; stopping that ssh process (its pid is in the pidfile) lifts the block.GET /api/link/status?routes=1addsroutes,route({name, index, since}) androuteBlockedto multi-route rows;ocx link status --jsonasks for it together with the link WebSocket counters (feat(link): the Home accepts the Responses WebSocket over a Remote Link, behind a closed release gate #8593) and the traffic counters (feat(link): measure the bytes each Remote Link carries and record request body size on log rows #8584), as?transport=1&traffic=1&routes=1, and validates all of them strictly; a row can carry every block. The default document is unchanged, except that an ssh client that is not OpenSSH reads as a failed multi-route link with reasonssh_client_unsupported. Route names stay on the Home's admin/dashboard surface (not in the Child-readable relay list) and out of diagnostics, history and warnings.SshRunner.rungains an opt-inAbortSignalwith one termination path (SIGTERM, SIGKILL after 2 s, settle within 2 s more) for timeout and abort; callers without a signal are unchanged.takeRouteProof(src/link/route-proof.ts), before anything else handles it.structure/remote-link.mdhas a new "Routes" section, including the readiness, return and orphan rules above.devat7e3cefd16(feat(link): measure the bytes each Remote Link carries and record request body size on log rows #8584 traffic counters, Fix nested model selectors in Pi exports #8287, test(sync): expect Pi's server-owned refresh after #8287 #8606, feat(link): compress Child request bodies with zstd for a Home that accepts it #8594 zstd request bodies, feat(link): the Home accepts the Responses WebSocket over a Remote Link, behind a closed release gate #8593 link WebSocket on the Home). Conflicts insrc/cli/link.ts,src/link/status-projection.ts,src/server/management/link-routes.tsand the link CLI tests were resolved by keeping both sides: the traffic block (?traffic=1) and the transport block (?transport=1) work as ondev, and the route fields as above.tests/cli/cli-link-activity.test.tsfromdevwas updated for the longer status query.Unchanged by default: links without
routestake the existing supervisor path (same argv, readiness, timings and orphan handling); the status document without?routes=1; exec operations (sync, update, observe, revoke) still use the alias.structure/providers/codex-installation-identity.mdandstructure/usage-client-attribution.mdownserve-options.tsandsrc/server/index/; they were reviewed and stay accurate.Verification
Tests written to fail first, run against the base before the change:
tests/clients/link-routes-store.test.ts,link-routes-argv.test.tsandlink-route-supervisor.test.tsfailed to load on the base, becauseeffectiveLinkRoutesandbuildRouteProbeArgvdid not exist.tests/clients/link-ssh-runner-abort.test.tsfailed 7 of 7: the runner had no abort contract.tests/clients/link-route-select.test.tsandlink-route-proof.test.tscover new modules.tests/clients/link-routes-status.test.ts).takeRouteProofis tested on real requests (header removed and reported, absent, not removable).Commands, with exit codes. The full set ran on
53c2f1209; the head adds only astructure/remote-link.mdwording fix (01f954df7), after whichstructure:check,privacy:scanand the structure, layout and repo-hygiene tests were run again (0, 0, 69 pass).bun run typecheck: 0bun run privacy:scan: 0bun run structure:check: 0bun run skill:surface:check: 0TMPDIR=/tmp bun scripts/test.ts --parallel=2overtests/clients/link-*,tests/server/link-*,tests/cli/*link*, every test file changed by feat(link): measure the bytes each Remote Link carries and record request body size on log rows #8584, Fix nested model selectors in Pi exports #8287, test(sync): expect Pi's server-owned refresh after #8287 #8606, feat(link): compress Child request bodies with zstd for a Home that accepts it #8594 and feat(link): the Home accepts the Responses WebSocket over a Remote Link, behind a closed release gate #8593 (includingcli-dispatchandsync-client-integrations),local-control-socket, the two test-layout tests,structure-ssot,file-size-ratchet,repo-hygiene,link-boundaryandcore-link-boundary(124 files): 1973 pass, 27 skip, 3 fail (exit 1). The 3 failures are the Rust ocx-child tests, which stop atcargo buildbecause the build machine has no default rustup toolchain.TMPDIR=/tmp OCX_OPENSSH_ROUTE_FIXTURE=1 bun test tests/clients/link-routes-openssh.test.ts: 3 pass (exit 0), OpenSSH 9.6p1. The fixture starts no daemon and opens no port: each route's ProxyCommand runs a privatesshd -ias the current user. It checks that the pinned probe passes, another key is refused without touching known_hosts, a granted-Rcarries bytes, a second session for the same port gets OpenSSH's failure line and exits, and a server without forwarding is refused. Thessh -Gcase runs without the opt-in.TMPDIR=/tmpwas used because the build machine's default temp path makeslocal-control-socket.test.ts's Unix socket path too long.NOT RUN: the full
bun run testsuite (the build machine allows focused files only; CI covers the rest); the three Rust ocx-child tests (tests/clients/link-ocx-child-e2e.test.tsand the Rust case oftests/clients/link-ocx-child-conformance.test.ts), which have no toolchain there; Windows and macOS behaviour, which uses the same argv builders (Windows Child proof via the existing .NET loopback request); the real-OpenSSH fixture in CI (opt-in).Design and code were reviewed by an independent reviewer: 14 plan rounds, then 7 code rounds, ending with no open P0–P2; the #8584 merge had a fresh final review with no findings. A fresh final review of
00c8abfa3found one P2 (the 64 KiB config cap) and seven P3s; all are answered in this head: fixed, documented (return rule, readiness deadline, orphan recovery) or, for the unbounded wait instopChild, kept as ondevwith the reason in the structure page. The author's own review of those fixes and of the #8593 merge had not finished at handover.Checklist
structure/remote-link.mdis updated; the user guide comes with the management commands in the stacked PR, since nothing user-facing writes routes yet.Implemented by Claude Opus 5.5 (anthropic/claude-opus-5-5) in GOAT through the Codex harness. Received as a git bundle from the VM that built it and pushed by the coordinator.