fix: policy-block visibility, sshx text read pipelining, and the non-leading sudo boundary - #81
Merged
Merged
Conversation
A line scanner asks the SFTP layer for a few kilobytes at a time, and pkg/sftp only overlaps round trips when it is handed a buffer larger than its packet size. Every read therefore cost a full round trip: an 8 MiB window over a ~40 ms link spent 83 s median in read latency while the remote transmitted ~800 KB/s. Read through a 1 MiB read-ahead aperture with UseConcurrentReads and a bounded in-flight request count. The aperture is bounded by the caller's byte budget, so --max-scan-bytes still caps what a scan pulls from the remote file, and reads smaller than a packet (the truncation probe) still go through untouched. SFTP sessions now come from newSFTPClient, which reuses the same connection with boundary-typed errors. Median for the reported command on the reporting host drops from 83.0 s to 26.5 s (88.0/77.9 s -> 23.8/29.1 s, alternating rounds) with identical bytes and lines scanned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A wide window over a slow link produced no output for a minute or more, so
callers timing out at 30 s could not tell a slow scan from a hang, and a
scan that stopped at its byte budget returned partial results without
saying so.
Narrate progress on stderr after a 3 s grace period, at most once per
second, with bytes, percentage, lines, elapsed and matches. A scan that
stops at --max-scan-bytes warns that its results are partial and names
--offset/--tail/--pattern/--max-scan-bytes as the ways to narrow it; a
slow scan closes with the same advice. Truncation reasons are a joined
list ("max_scan_bytes,max_hits"), so the warning tests membership instead
of string equality.
stats gains expected_scan_bytes, the byte budget of the window, so a
caller can size a scan and spot a partial one before trusting
total_hits_exact. The field is additive and omitted for journal scans.
stdout still carries exactly one sshx.text.v1 document: progress and
advice are stderr-only.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A policy block carries its reason in the --json document, so a caller that prints only exit_code/stdout/stderr read "exit_code=-1" with two empty streams and concluded the refusal was silent. Mirror the decision and the flattened reason to stderr in --json mode. stdout is unchanged: the same one-document payload, with the same block fields. The mirror is emitted once, and only for a real admission block. Sudo auto-fill still rewrites a leading sudo only. That boundary was correct but invisible: `cd /data/app && sudo docker compose up -d` died with "sudo: a password is required" 29 times in one session with no explanation. Warn before connecting when a command runs sudo in command position after a segment or wrapper, and explain the refusal afterwards when the remote output shows a sudo password prompt. Both hints write straight to stderr, because a caller reading a failed run may have quieted diagnostics; the run path names the failing target. The auto-fill scope itself is unchanged. commandTokenIndex is extracted from commandInPosition so sudo position detection shares the shell segmentation rules, and sudo gets its own walker because commandWrappers already skips it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record the three session-analysis fixes: the machine-readable block predicate (exit_code=-1 + error_kind=blocked + phase=admission + executed=false) with the guarded sshx sql --docker= alternative, the monitoring contract of sshx text (stderr-only progress, partial-scan warning, expected_scan_bytes), and the auto-fill boundary that only the first token of a command can be rewritten. Covers README, README_CN, usage.go, AGENT.md section 7.4 and the Unreleased CHANGELOG section, including the measured 83.0 s -> 26.5 s scan timing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Guard the reported defects end to end against a real SSH/SFTP server: a window larger than a read packet stays complete, a budget-limited scan warns on stderr with a joined reason while stdout keeps one JSON document, a fast scan stays quiet, a blocked command mirrors its reason without touching stdio, and a non-leading sudo is warned about before connecting and explained after the remote refuses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Route every best-effort stderr note (block mirror, sudo hints, scan progress, truncation warning) through one helper that checks the write error and records it at debug level, so a failed diagnostic write can never change a command outcome. The helper writes only to the injected writer, keeping --json stdout pure. Found by golangci-lint's errcheck; the CI lint job never got this far because config verification failed first. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The lint job pins golangci-lint `version: latest`, which resolved to 2.13.2 and now rejects the retired top-level `issues.exclude-rules` key, so `golangci-lint config verify` aborts before any file is linted. The job is red on main for the same reason. Move the rule to `linters.exclusions.rules` (the v2 location). The rule itself is unchanged, `config verify` passes, and the migrated config still reports the same findings: 0 issues on main and the 6 errcheck findings fixed in the previous commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Sep 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the three defects confirmed by the read-only session-usage analysis of 2026-09-20 (evidence in #80), plus the documentation that kept the sudo boundary invisible.
Security semantics are unchanged: a direct
psqlis still blocked, and--jsonstdout still carries exactly one document.What changed
sshx textwas slow and silent (P1). A line scanner asks the SFTP layer for a few kilobytes, and pkg/sftp only overlaps round trips when handed a buffer larger than its packet size — so every read cost a full round trip. Scans now read through a 1 MiB read-ahead aperture withUseConcurrentReadsand a bounded in-flight count, and SFTP sessions come fromnewSFTPClient(same connection, boundary-typed errors).Same command, same host, alternating rounds, pristine
390caffbuild vs this branch:Identical work in every run (
bytes=8388608, ~46 980 lines,truncated_reason=max_scan_bytes). The remaining cost is the link (~800 KB/s, ~1 s per 32 KiB request), not the client.--max-scan-bytesstill bounds what a scan pulls: the same budget bounds the aperture, and the one-byte truncation probe bypasses it.A long scan looked like a hang (P1).
sshx textnow narrates progress on stderr after a 3 s grace period (bytes, percentage, lines, elapsed, matches), warns when a scan stops at its byte budget and names the ways to narrow it, and closes a slow scan with the same advice.sshx.text.v1gains the additivestats.expected_scan_bytes. stdout keeps exactly one JSON document.A policy block looked like a silent refusal (P1).
--jsonmode now mirrors the block decision and the flattened reason to stderr, so a caller that prints only the streams sees why nothing ran:Non-leading
sudo(P2). Auto-fill still rewrites a leadingsudoonly, but the boundary is now announced: sshx warns before connecting when a command runs sudo in command position after a segment or wrapper, and explains the refusal when the remote reports a password prompt — suggestingsudo sh -c "<command>".cd /data/app && sudo docker compose up -dfailed 29 times in the reporting session with no explanation.Files
internal/sshclient/sftp_read.go,text.go,transport.go,remote_state.gointernal/app/text_progress.go,internal/app/text.go,internal/textsafe/{scan,types,help}.go,docs/text.mdinternal/app/lifecycle.go,internal/app/app.go,internal/app/run.go,internal/sshclient/validate*.goREADME.md,README_CN.md,internal/app/usage.go,AGENT.md,CHANGELOG.mdBugfix reflection
Lint gate
The CI lint job pins golangci-lint
version: latest, which now resolves to 2.13.2 andrejects the retired top-level
issues.exclude-ruleskey, soconfig verifyaborts beforea single file is linted. That job is red on
mainfor the same reason. Two commits dealwith it:
chore(ci): the rule moves to its v2 home,linters.exclusions.rules. Rule and effectare unchanged —
config verifypasses, and the migrated config still reports 0 issueson
mainand the 6 errcheck findings below. Drop this commit if the config migrationshould land separately, but the lint job cannot run without it.
fix(app): running the linter locally at CI's version surfaced 6 uncheckedfmt.Fprintf/Fprintlndiagnostic writes in this branch's new code — the CI job neverreached them because verification failed first. They now go through one best-effort
helper that checks the write error and records a failure at debug level, so a failed
diagnostic write cannot change a command outcome and
--jsonstdout stays pure.Local gate at golangci-lint 2.13.2 with the repository config:
Worth considering: pin the action's
versioninstead oflatestso a linter releasecannot switch the gate off again without a code change.
Notes
truncated_reasontomax_scan_bytesexactly, which silently skipped the warning for the real-world joined form (max_scan_bytes,max_hits). Fixed withtextsafe.HasTruncationReason, and both a unit and an E2E test use a joined reason. This is why running the live reproduction, not just the unit tests, was worth it.TestApplySudoScriptEvidenceAndCleanupfails at pristine390cafftoo; out of scope here.