Skip to content

fix(statusline): inject plan-usage telemetry via ephemeral CLI flag, never disk - #361

Open
timkjr wants to merge 4 commits into
Ark0N:masterfrom
timkjr:fix/statusline-injection-opt-out
Open

fix(statusline): inject plan-usage telemetry via ephemeral CLI flag, never disk#361
timkjr wants to merge 4 commits into
Ark0N:masterfrom
timkjr:fix/statusline-injection-opt-out

Conversation

@timkjr

@timkjr timkjr commented Aug 30, 2026

Copy link
Copy Markdown

Summary

The plan-usage chip's statusLine exporter was written into the case's .claude/settings.local.json. That file-based statusLine took precedence over the user's own global/project statusline for any claude invocation in that directory — including entirely outside Codeman — with no disclosure in the App Settings UI (labeled only as a header-display toggle) and no way to remove it once written (the removal code path was dead code: nothing ever called applyStatusLineConfig(path, false)).

Fixes this by injecting the exporter as an ephemeral claude --settings '{"statusLine":{...}}' CLI flag, resolved fresh at spawn time and merged with effort/ultracode into one --settings object (Claude Code accepts only one --settings flag per invocation). Never touches disk, so a plain claude run outside Codeman is completely untouched. Self-healing: a legacy disk-written exporter from an older build is stripped the first time a session starts in that workspace again. Still respects a user's own hand-authored statusline — it's never overridden.

Bonus: since the exporter no longer lives in a fixed file, it can now wrap the user's own real statusline instead of just backing off. findEffectiveUserStatusLineCommand() walks Claude Code's actual settings precedence (project settings.local.json → project settings.json → user ~/.claude/settings.json) to find whatever's really configured, and the shared exporter script fires the telemetry POST in a background subshell (confirmed non-blocking, ~4ms, even against an unreachable endpoint) then execs the user's real command with the same stdin — so the user keeps seeing their own statusline, and the plan-usage chip still gets its data.

A subtler bug found mid-fix

The exporter's command legitimately depends on $CODEMAN_SESSION_ID / $CODEMAN_API_URL / $CODEMAN_HOOK_SECRET_FILE and an internal $INPUT, all meant to be expanded only when Claude Code itself executes the statusline, using the pane's tmux setenv'd environment. An early version of this fix passed that command as literal text through --settings, which routed it through execSync's own implicit /bin/sh -c (tmux respawn-pane's bash -c "..." wrapper) before Claude Code ever saw it — POSIX double quotes don't suppress $ expansion, so those vars got expanded prematurely against the server process's own environment (unset there), producing malformed JSON that printed as literal error text in the statusline.

Fixed by writing the exporter as a real, shared script file (ensureStatusLineExporterScript, marker-versioned so stale copies self-heal) and passing only its bare path via --settings — a path has no shell metacharacters for any intermediate shell layer to mangle.

Test plan

  • npm test (full CI gate) green throughout
  • New unit tests: test/hooks-config.test.ts (resolveStatusLineCliCommand, findEffectiveUserStatusLineCommand), test/statusline-cli-flag.test.ts (real-shell-execution based, not hand-rolled unescaping)
  • Verified live against a real Claude CLI on an isolated tmux socket (outside the app, own statusline shown untouched; inside the app, telemetry flows)
  • Verified via direct execSync reproduction of the exact nested shell wrapping createSession/respawnPane use, with an arg-echoing stand-in for claude, before trusting the fix
  • Confirmed the background telemetry POST never blocks statusline rendering (timed against an unreachable endpoint: ~4ms)
  • Deployed to my own production instance; confirmed both in-app and outside-app behavior, including a remote SSH session (unaffected either way — remote sessions were never in scope for this injection)

@Ark0N

Ark0N commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Thanks for this, and for the shell-layer analysis, which is careful and correct. Writing Codeman's exporter into the user's settings.local.json was always the weaker half of the plan-usage chip, and moving it to an ephemeral --settings flag is the right direction. There is one design question I want to settle with you before you rework anything, because the answer decides how much of the rest is needed.

The question: create-time-only, or sticky? (src/web/routes/system-routes.ts:1028). Under the disk mechanism, one enable covered every later session in that workspace regardless of creation path: cron (cron-service.ts:428), the orchestrator (plan-orchestrator.ts:438 and :604), the Ralph API (ralph-routes.ts:335), quick-start (session-routes.ts:2878) and legacy scheduled runs (server.ts:1912). Turning the chip on also started telemetry in already-running sessions, because Claude Code re-reads settings.local.json live. After this PR, telemetry flows only from POST /api/sessions, so a user whose running sessions are cron- or orchestrator-launched sees a dark chip, and switching the chip on does nothing until a new UI-created session exists. The frontend still sends the action field (settings-ui.js:2288) and the comments at settings-ui.js:2249, session-ui.js:951 and server.ts:1407 all still describe the old flow.

Two ways out:

(a) Keep sticky semantics server-side: persist the PUT action as a settings.json flag consulted by every claude create, respawn and restore path. This also resolves the next item without needing a per-session field at all.
(b) Accept create-time-only and say so, in CLAUDE.md, docs/architecture-invariants.md, docs/usage-limits-display-plan.md:118 and the App Settings hint.

I lean toward (a), because "turn the chip on and nothing happens" is a bad experience and the disk mechanism did not have it. Your call if you see a reason (b) is better.

Then, whichever way that goes:

1. Rebase onto master's CLI registry (src/tmux-manager.ts:954). git merge-tree conflicts in src/tmux-manager.ts and src/web/routes/system-routes.ts. Master's 4830e662 (#347) deleted the per-mode buildSpawnCommand ladder this PR edits: claude's --effort and --settings are now rendered by the registry argv engine from the effortLevel/effortSettingsJson engine values (stock.ts:142, session-cli-registry-bridge.ts), per-CLI env setup moved into _configureCliEnv() driven by registry env.configSetenv, and master's settings handler gates on getCli(mode)?.capabilities.statusLineTelemetry. The port: add statusLineCommand to SpawnBridgeOptions, merge it with the ultracode blob into one settings-JSON engine value in the bridge, keep the argv engine as the sole renderer (its header forbids shell text in config, which a bare path satisfies), and give the user-statusline setenv a home beside _configureCliEnv.

2. statusLineTelemetry is not persisted, so a restart turns telemetry off on the next respawn (src/session.ts:1400). toState() emits effort but not this; SessionState has no such field; restoreMuxSessions restores effort only. After any Codeman restart, which is every deploy, a restored session has _statusLineTelemetry === undefined, and its next respawn (Ralph cycle, /clear respawn, PTY-exit restart) calls resolveStatusLineCliCommand(dir, false), which strips the legacy disk exporter and injects nothing. The chip goes dark for that session permanently. The disk mechanism survived restarts by construction. Option (a) above makes this moot; under (b) it needs the field in SessionState, emitted in toState(), restored next to effort, with a round-trip test like the effort one.

3. The in-terminal footer print-through is dropped (src/hooks-config.ts:898). /api/status-telemetry returns formatSessionStatusText(...) (model, tokens, context %) precisely so the exporter can print it as the footer, and the route's own @fileoverview, CLAUDE.md:198 and docs/architecture-invariants.md:91 all say so. The new script sends curl's stdout to /dev/null and prints the literal codeman whenever the user has no statusline of their own, which is the common case. In the no-user-statusline branch, run the POST in the foreground and print its body with || echo codeman as the fallback; background it only in the wrap branch, where the user's command owns the footer.

4. The background subshell inherits stdout, so a host reading to EOF still waits for curl (src/hooks-config.ts:899). Reproduced with your generated script and sleep 5 standing in for curl: sh script | cat takes 5.005 s, and from Node with piped stdio the exit event fires at 8 ms but close (stdout EOF) at 5010 ms. With ) >/dev/null 2>&1 </dev/null & close fires at 9 ms. Your ~4 ms measurement matches a tty stdout. Whether Claude Code waits for exit or for EOF I could not verify from its source, but every ordinary Node pattern waits for EOF and the fix is one redirection. Please also add --max-time to the curl so a hung (rather than refused) Codeman cannot wedge the render; the old exporter lacked it too.

Smaller: the docs and registries did not move with the code.

I would settle the sticky question first, then the rebase, the footer and the redirection are each small once the shape is agreed, and the rest are one-liners. The tests you wrote are real ones, so this is worth a round trip rather than a rewrite.

timkjr and others added 3 commits September 7, 2026 19:15
…never disk

Codeman's plan-usage chip wrote a statusLine.command into the case's
.claude/settings.local.json to receive Claude Code's rate_limits blob.
That file-based statusLine took precedence over the user's own
global/project statusline for ANY `claude` run in that directory,
including entirely outside Codeman, with no disclosure in the App
Settings UI (labeled only as a header-display toggle) and no way to
remove it once written (the removal code path was unreachable dead
code — nothing ever called it with false).

Replace the disk write with an EPHEMERAL `claude --settings
'{"statusLine":{...}}'` CLI flag, resolved fresh at spawn time
(resolveStatusLineCliCommand in hooks-config.ts) and merged with
effort/ultracode into one --settings object (buildClaudeSettingsFlag
in tmux-manager.ts, since Claude Code accepts only one --settings
flag). Never touches disk, so a plain `claude` run outside Codeman is
untouched. Self-healing: any legacy disk-written exporter from an
older build is stripped the first time a session starts in that
workspace again. Still respects a user's own hand-authored statusLine
(skips the flag entirely rather than overriding it).

Mid-fix bug found and fixed: the exporter's command legitimately
depends on $CODEMAN_SESSION_ID/$CODEMAN_API_URL/$CODEMAN_HOOK_SECRET_FILE
and an internal $INPUT, all meant to be expanded only when Claude Code
itself executes the statusline, using the pane's tmux-setenv'd
environment. Passing that text through --settings routed it through
execSync's own implicit /bin/sh -c first (tmux respawn-pane's
`bash -c "..."` wrapper) — POSIX double quotes don't suppress $
expansion, so those vars got expanded prematurely against the
server's own environment (unset there), producing malformed JSON that
printed as literal error text in the statusline. Fixed by writing the
exporter as a real, shared script file (ensureStatusLineExporterScript,
marker-versioned so stale copies self-heal) and passing only its bare
path via --settings — nothing for any intermediate shell to mangle.
Verified against a real Claude CLI on an isolated tmux socket, and via
direct execSync reproduction of the exact nested wrapping
createSession/respawnPane use.

A hard "never inject, even ephemerally" kill-switch was added and then
removed in the same pass: with the disk-leak fixed, disabling
injection only cost the plan-usage telemetry the feature exists to
provide, for no remaining benefit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015GyMnFWnUzc41TDeHg9juW
…ping it

Now that the exporter no longer lives in a fixed per-case file, it can
compose with the user's actual configured statusline rather than just
backing off when one is found.

findEffectiveUserStatusLineCommand() walks Claude Code's own settings
precedence for a workspace: project-local .claude/settings.local.json
> project-shared .claude/settings.json > the user's global
~/.claude/settings.json. A legacy Codeman-marked entry left behind in
the project's own settings.local.json is never treated as a real user
command — it's skipped and precedence continues to the next layer.

The shared exporter script (bumped to a V2 marker so stale copies
self-heal) now fires the telemetry POST in a background subshell —
its own stdout/stderr discarded so nothing leaks into the visible
statusline, and confirmed non-blocking (~4ms, even against an
unreachable endpoint) — then, if the pane's environment carries
CODEMAN_USER_STATUSLINE_CMD, feeds it the same stdin blob and relays
its stdout as ours. Otherwise it falls back to the plain "codeman"
marker as before.

The discovered command is threaded to the pane via `tmux setenv
CODEMAN_USER_STATUSLINE_CMD` (_configureStatusLineUserCommand) rather
than embedded in the spawn command line, for the same
premature-shell-expansion reason as the parent commit: tmux stores a
setenv value verbatim and never re-parses it, so once shellescape()d
for that one command, the command's own $/quotes survive untouched
into the pane's environment.

Verified live via direct shell execution of the generated script
(both branches: fallback and user-command wrapping) before deploy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015GyMnFWnUzc41TDeHg9juW
…OF fix

Responds to Ark0N's review round on the ephemeral-CLI-flag statusline
injection rework:

- Rebase-detail fixes: registry-gated telemetry eligibility via
  getCli(mode)?.capabilities.statusLineTelemetry instead of a hardcoded
  mode === 'claude' check, using the capability flag master's CLI-registry
  refactor already declares for exactly this purpose.

- Design question settled: sticky (a). Rather than persisting the toggle
  as a new field and threading it through every session-creation path
  (cron, Ralph Loop API, quick-start), eliminated the per-session field
  entirely. readPlanUsageTelemetryEnabled() (hooks-config.ts) reads the
  existing showPlanUsageLimits setting fresh from settings.json at every
  claude create/respawn (TmuxManager.createSession/respawnPane) - no
  per-session state to survive a restart, and it applies uniformly to
  every creation path for free, since they all flow through the same
  TmuxManager methods.

  This required fixing a real bug found along the way: showPlanUsageLimits
  was not actually round-tripping through settings.json on save -
  settings-ui.js explicitly excluded it from the PUT body as a pure
  per-device display key. It now flows through normally (both true and
  false); the load-side per-device merge behavior is unchanged.

  Removed entirely as a result: the statusLineTelemetry field from
  CreateSessionSchema/SettingsUpdateSchema, CreateSessionOptions/
  RespawnPaneOptions, Session._statusLineTelemetry (this is what makes
  the restart-persistence bug moot rather than patched), and the
  frontend send sites.

- Footer print-through restored: the no-user-statusline branch of the
  exporter script now runs the telemetry POST in the foreground so its
  own stdout becomes the in-terminal footer, falling back to a plain
  "codeman" marker only on curl failure.

- Background-subshell EOF fix: the wrap-a-real-statusline branch closes
  stdin too, not just stdout/stderr (`>/dev/null 2>&1 </dev/null &`) -
  the un-redirected subshell process itself, not curl, was what held a
  reader-to-EOF's pipe open for however long curl took to finish. Added
  curl --max-time 5 so a hung (not just refused) Codeman cannot wedge
  the render.

Tests: real-shell-execution tests for the footer/EOF fixes (fake curl
stand-in on PATH, real sh subprocess spawns, real elapsed-time
measurements - verified non-vacuous against a hand-reconstructed
old-style script), unit tests for readPlanUsageTelemetryEnabled.
Adapted two existing tests whose payloads referenced the removed field.
Fixed during independent code review: a stray indentation break and a
test exercising the wrong (legacy) exporter code path.

Docs synced: CLAUDE.md, docs/usage-limits-display-plan.md (old
disk-based section marked superseded, kept for history),
docs/architecture-invariants.md.

Full suite green: 352 files, 6780 passed, 12 skipped, 0 failed.
tsc/lint/format:check/frontend-syntax all clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@timkjr
timkjr force-pushed the fix/statusline-injection-opt-out branch from 7aff633 to d5b75af Compare September 8, 2026 01:41
@timkjr

timkjr commented Sep 8, 2026

Copy link
Copy Markdown
Author

Thanks for the detailed breakdown — went with (a), sticky. Turned out simpler than expected: rather than persisting the toggle as its own field and threading it through cron/the Ralph API/quick-start, I reused showPlanUsageLimits (which, it turns out, wasn't actually round-tripping through settings.json before this — found and fixed that too) and read it fresh at every claude create/respawn. That resolves your #2 for free — nothing per-session to lose on a restart — and every creation path gets it automatically since they all go through the same TmuxManager methods, no wiring needed anywhere else.

Rebased onto master's CLI-registry refactor (#347): statusLineCommand merges into the same --settings JSON object as effort/ultracode in the bridge, argv engine stays the sole renderer. Also swapped the hardcoded mode === 'claude' check for capabilities.statusLineTelemetry, since that flag was already there for exactly this and had lost its only consumer during the rebase.

#3 and #4 are both fixed — foreground POST with real print-through in the no-wrap branch, >/dev/null 2>&1 </dev/null & (closing stdin too) plus --max-time 5 in the wrap branch. Added real-shell-execution tests for both (fake curl on PATH, actual sh spawns, measured elapsed time) and confirmed they'd have failed against the old script.

Docs are synced (CLAUDE.md, usage-limits-display-plan.md, architecture-invariants.md). Full suite's green: 352 files, 6780 tests, 0 failed.

planUsageChipEnabled() (settings-ui.js) shows the header chip and the App
Settings checkbox as already ON whenever showPlanUsageLimits has never been
set — a discoverability default from 1.9.3. readPlanUsageTelemetryEnabled()
(hooks-config.ts) deliberately treats an absent key as "no telemetry" — a
privacy default, pinned by its own unit tests (never POST usage data
without an explicit persisted yes). Nothing reconciled those two
independent guesses, so a fresh install showed a checked box that silently
collected nothing until the user opened Settings and hit Save at least
once.

Verified live: an install that had never touched this setting had no
showPlanUsageLimits key in settings.json at all, and its running Claude
process's argv carried no --settings flag — zero telemetry ever collected
despite the chip rendering as enabled.

GET /api/settings now persists the resolved default (true) the first time
the key is truly absent — not explicit false — so "chip visible" and
"telemetry collected" become the same fact. readPlanUsageTelemetryEnabled's
own absent-means-false contract is untouched; after this runs once the key
is never absent again, so that branch stays correct in isolation while
being unreachable in practice for any install that has ever called this
route. An explicit false set afterward is respected forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@timkjr

timkjr commented Sep 11, 2026

Copy link
Copy Markdown
Author

Found and fixed a follow-up gap in this one, verified live rather than just in the diff.

The sticky-collection design you signed off on works correctly — readPlanUsageTelemetryEnabled() really does read showPlanUsageLimits fresh at every create/respawn. But any install — new or long-running, it doesn't matter — that has never triggered a PUT /api/settings save since this mechanism went live has the key completely absent from settings.json, because nothing in normal usage calls that route on its own. planUsageChipEnabled() (client-side) defaults an absent key to true, so the chip renders as already on — but readPlanUsageTelemetryEnabled() deliberately defaults absent to false (correct, tested behavior — never POST usage data without an explicit yes). Nothing reconciles those two guesses. I hit this on my own long-running install: chip visible, settings.json had no showPlanUsageLimits key at all, and the running Claude process's actual argv had no --settings flag — zero telemetry, ever, despite the UI claiming it was on. Every existing user who upgrades to pick up this PR hits the same thing unless they happen to resave Settings.

Fix: GET /api/settings now persists the resolved default (true) the very first time it sees the key is truly absent — not explicit false. readPlanUsageTelemetryEnabled()'s own absent-means-false contract and its existing tests are untouched; after the first read the key is never absent again, so that branch stays correct in isolation while becoming unreachable in practice for any install that's ever loaded the app. An explicit false you set afterward is respected forever — this only fires on true absence.

Added round-trip route tests for all four cases (absent + file missing entirely, absent + file exists with other keys, explicit false preserved, explicit true left alone) and updated the two existing GET /api/settings tests that pinned the old raw-passthrough behavior. Full suite green: 357 files, 6840 tests, 0 failed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants