Skip to content

fix(auto-reply): harden send gates and split main.rs and context - #49

Merged
twoimo merged 76 commits into
mainfrom
stability/1-refactor-and-harden
Nov 16, 2026
Merged

twoimo merged 76 commits into
mainfrom
stability/1-refactor-and-harden

Conversation

@twoimo

@twoimo twoimo commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Why

Alden sends real KakaoTalk messages, so a gate bug can message the wrong person, and the three largest files (src/main.rs, src/context/mod.rs, plus several 2–5k-line modules) were hard to change safely. This PR splits the largest Rust modules without behavior change and fixes confirmed defects in the send gate, network paths, desktop process control, and the Python process runtime. Each fix starts with a test that failed on the old code.

What changed

  • src/main.rs (11,953 → 3,039 lines) keeps the clap types and dispatch; handlers move to src/commands/ (auto_reply/ gets one submodule per concern). src/context/mod.rs (9,728 → 74 lines) becomes 13 noun-named submodules.
  • Send gate: a catalog room without an integer chat_id no longer allows every room name; the AX composer is written only after a positive empty read, and a submit is never re-applied after the composer was seen empty.
  • Stability: char-safe truncation of Korean text, bounded chat paging and LOCO timeouts, atomic and locked state.json, process-group stop and full abort coverage in the desktop bridge, GeekNews-only rooms keep db-watch off, supervisor crash cleanup, no Terminal hiding or window scripting, CI runs 8 more Python modules and releases gate on clippy.

Scope

  • Covers behavior-preserving splits and test-first fixes. Desktop TS and the Python AI modules also get a clean-code pass.
  • Leaves the worker file, the double-send fix and hermetic runtime tests to the next PRs in this stack.

Blast Radius

CLI help output, exit codes and JSON are unchanged (all 80 command paths diffed). The fixes change the live send path only toward refusing: an unreadable composer or an id-less catalog room now blocks the send. Python changes are source-only until a new runtime is baked.

Verification

🤖 Generated with Claude Code

twoimo and others added 30 commits November 16, 2026 01:03
Characterize the safety contract of the context store before moving any
code. Read APIs on a missing database return the migration-required error
without creating the file or its directory, a symlinked database is
refused, and the read-only connection rejects writes. Opening refuses a
symlinked WAL sidecar without touching its target. A CSV reindex
truncates the WAL and scrubs replaced text even with a concurrent
connection open. The reply-queue loader never follows a symlink or
creates a missing queue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Format-only. Stripping whitespace, the diff is nine removed and three
added trailing commas plus braces around closure bodies, so the module
no longer contributes to cargo fmt --check failures before the split.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove code nothing in the repo reaches:
- The non-recipient "excluding live events" bundle family
  (context_reply_bundle_excluding_live_event[s][_json]). The CLI bails on
  that argument shape, so only the recipient variant can exclude rows.
  context_reply_bundle now carries the body of its former
  context_reply_bundle_internal, and context_ids_for_excluded_live_events
  takes the exclusions directly instead of an always-Some Option.
- confirmed_self_sends_from_decisions and REPLY_DECISION_STATUSES, which
  had zero references.
- Test-only one-line wrappers context_reply_bundle_for_recipient[_json]
  and rebuild_context_index. Their tests now call the wrapped function
  with the same arguments.
- stable_provenance_id, a private alias of provenance_id.
- search_with_connection and reply_decision_search_with_connection,
  each with one caller.
- Two empty if-blocks that held only a comment, and the empty index,
  live and reply_bundle stub modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Children started without a caller token (mlx-server-launch, model-set,
model-prepare, room-upsert, knowledge actions) were never registered, so a
global abort could not reach them during their long timeouts. Register every
child under an internal key that caller tokens cannot name or cancel.

Adds fixture-backed bridge tests that run the real spawn path with a
synthetic python3 child and assert the abort returns and the child is reaped.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
Delete symbols with no caller anywhere in the repo (scripts, tests,
desktop, macOS, manifests): ALIAS_DICTIONARY_VERSION, VECTOR_CANDIDATE_MIN,
QWEN38_27B_MIN_MEMORY_GB, ModelResidencyManager.cancel and
reset_after_human_resume, detect_available_engines, is_audited_gold.
Drop the unused aliases binding in the graph context builder and keep its
json.loads, which still rejects a corrupt row. Remove an f-string with no
placeholders. No behavior change.

Co-Authored-By: GPT-6.1 Sol via opencodex <noreply@openai.com>
Timeouts and cancels killed only the direct child, so a descendant that
inherited stdout kept the pipe open and the reader join stretched the timeout
until it exited. Spawn each child in its own process group, SIGTERM the group,
allow a short grace, then SIGKILL the group. Bound every pipe drain and detach
a thread that a surviving descendant still holds instead of joining it.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
open_db now reads as the file lifecycle (directory, permissions, open,
busy timeout, WAL mode) around one migrate(conn, was_missing) call. The
legacy rename, pragmas, DDL, additive columns, retrieval marker and the
three data migrations run in the same order as before. This is the seam
the store/schema split needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run_browser_tool and fetch_settings_action began their job before the child
registered its token. A duplicate token was refused by the registry, but the
caller still finished the job, removing the entry of the call that was still
running. Begin the job only after registration succeeds, so refused calls
never touch the registry.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
start_voice_session spawned the interpreter and dropped the Child without
waiting, so each session left a zombie once it exited. Hand every detached
child to one background wait that collects its exit status.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
The marker temp name was derived from the pid alone, so two writers in one
process raced on create_new and the second failed with a state error. Add a
process-wide sequence number to the temp name.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
Three diagnostic previews sliced strings by byte index and panicked when the cut landed inside a multi-byte character: the login response preview (200 bytes), the renew_token field dump (60 bytes) and the CHATINFO field dump (100 bytes). A 100-character Korean value was enough to crash login error reporting. All three now go through util::truncate, which counts characters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove the parenthesised (YYYY-MM-DD ...) review dates that narrate
history git already records, keeping each sentence's why. No code
or string literal changed.

Co-Authored-By: GPT-6.1 Sol via opencodex <noreply@openai.com>
Quitting during a model swap or other bridge job left the Python child
running. Add an exit hook that gives model swaps their cooperative marker and
a short grace, then hard-stops whatever is still registered. It deliberately
does not call global_abort: that writes the emergency latch, which refuses
every later job until a human resumes.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
Eight test modules passed on a fresh checkout with Python 3.11, no user
data, no network, and no host services, but CI never ran them. They add
171 tests and about 40 seconds to the desktop job.

tests.test_auto_reply_menubar stays out because it needs a frozen .pyc.
The full tests.test_auto_reply_cli_runtime module stays out because 73 of
its 348 tests fail on a fresh checkout. The desktop crate is
rustfmt-clean, so its format is now checked. The CLI crate is not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release build ran only cargo test, so a tag could ship code that main
rejects. It now runs the same clippy step as the macOS CI job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The checklist listed a cargo fmt gate that CI does not run. It also
used clippy without --all-targets, and it left out the launchd harness,
the Python tests, and the desktop steps. It now lists the same commands
as ci.yml and reads the Python module list from ci.yml. Release step 5
now names the macos_build job instead of a verify job that does not
exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d cancel flag

The comment cleanup removed six (사용자 지시) markers. They record that a
behavior was the owner's product decision, which the code cannot show, so
they come back without the dates. Removing cancel() left _cancelled always
False; _is_cancelled still reads cancel_check, so the field goes too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
get_all_chats looped for as long as the server returned a non-None cursor, so a server that repeated a cursor made the CLI hammer talk-pilsner forever. Pagination now errors when a cursor repeats and after 100 pages, the same bound get_all_messages uses. The loop takes a fetch closure so it can be tested without the network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
full_connect cast server-supplied i32 ports with 'as u16', so 70000 became 4464 and -1 became 65535, and it connected to the wrong port. checkin already range-checked the port for its log line but full_connect re-parsed it unchecked, so the log and the real connection could disagree. One parser now rejects ports outside 1..=65535: booking drops them, and checkin falls back to the default port its log already reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hannel

Every catch in auto_reply_metrics.py was `except BaseException`, so a
SystemExit or KeyboardInterrupt raised while a metric was being written
(for example from a SIGTERM handler that calls sys.exit) was swallowed
and the decorated service function returned normally. record() and
Measurement.__exit__ run on every @perf.timed call, so a shutdown that
lands during the metrics file write was silently lost.

Narrow all seven catches to `except Exception`. Ordinary metrics I/O
errors still never reach the caller; process-control exceptions do.

Before: test_shutdown_raised_during_metric_write_reaches_the_caller
failed with "SystemExit not raised". After: both metrics tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A GeekNews-only room must run the reply worker without the per-account
db-watch. main() only applied that rule when db_ready() was true at
startup, and then reset database_started to False. The main loop treats
database_started=False as "DB not ready yet": on the first tick db_ready()
answered true, so the loop started db-watch and set
OPENKAKAO_DB_READY=1 and OPENKAKAO_AUTO_REPLY_ENABLED=1 anyway. A room
whose DB was unready at startup skipped the rule entirely and was
promoted the same way later.

Decide geeknews_only once, before the DB probe, publish it as the
database reason, and make the promotion branch respect it.

Before: both GeekNews-only main() tests failed with db-watch in the
spawned scripts. After: they pass, and the reply-room control still
starts db-watch once the DB becomes ready.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
booking, connect, send_command and loco_upload waited without a deadline, so a server that accepted the socket and never answered hung the CLI forever. Only checkin (10 s) and Legacy frame reassembly (30 s) were bounded. Every one-shot exchange (booking and checkin) and every connection setup now has a 10 s deadline, and every reply wait on an open connection has a 30 s deadline. A timed-out command marks the connection dirty so the next call reconnects instead of reading a half-consumed stream. The public recv_packet used by watch loops stays unbounded on purpose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The inline test module becomes src/context/tests.rs at the same module
path, context::tests, so test names are unchanged. The body is identical
once whitespace is ignored. Lines are dedented one level except inside
string literals, and rustfmt rejoined lines that now fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main.ts had one test for its 600 lines, and the snapshot parser was only
checked field by field. These characterization tests record what the
settings window, the panel boot and the bridge parser do today, so the
clean-code pass that follows can be checked against them.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… reports

The provider tests asserted status and reason only. Pin the whole report
for every failure path of both paper providers, the fixed shapes of the
DPO and runtime-probe rejections, and the voice file pipeline's JSON
report, exit codes and WAV fallback writer.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
mod.rs becomes the module root: a module map, declarations and
re-exports. Every item moves unchanged into the submodule that owns its
body of knowledge, and common.rs dissolves into the owners of its
constants:

- store: the database file, its permissions, WAL and read-only open
- schema: tables, the retrieval-index marker and all migrations
- embedding: the message vector and its byte encoding
- csv_import: exported chat CSV import
- live: live local-DB ingest, per-source sync state, authoritative source
- self_send: confirmed self-sends and the bot-sent style repair
- style: owner and recipient style classification and profiles
- topics: the topic lexicon and message topic rows
- response_time: the fitted reply-delay distribution
- reply_decision: decision records and their status order
- search: keyword, vector, topic and hybrid retrieval
- reply_bundle: the versioned reply bundle
- redact: provenance hashing and path redaction

Every public item keeps its openkakao_cli::context path through a
re-export. Items used across submodules become pub(super), so nothing
outside context gains access. With whitespace, use lines, visibility
qualifiers and trailing commas ignored, all 200 items are token-identical
to the pre-split source, and tests.rs gains only imports plus its one
test-only helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main() had no handler around the child start sequence or the main loop.
An exception there (OSError ENOSPC from write_status, EAGAIN from
start()) escaped main() and left apple-watch, db-watch and the reply
worker running until something outside the supervisor swept its process
group.

Wrap the body in main(): on any Exception, print the traceback as
before and run stop(reason="supervisor_error", exit_code=1), which
terminates, kills and reaps every started child and publishes the stop.
SystemExit and KeyboardInterrupt are not Exception and pass through
unchanged.

Before: both cleanup tests failed with the started FakeChild entries
still running. After: every started child is terminated and main() exits
with status 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It takes a database path and only opens it, so it sits with the other
path-level entry points. schema no longer imports store, which leaves
every schema function working on a Connection and removes the only
store/schema import cycle. The public path is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The voice start probe lists same-user Python processes with ps, then reads
each argv. A process that exits between the two calls makes macOS answer
EINVAL, not ESRCH, so the probe failed closed with voice_session_process_check_failed.
Treat a PID that is gone or a zombie as exited; a live process whose argv
cannot be read still blocks the start.

Found while gating the bridge fixes: the voice reap test intermittently failed
under concurrent test processes.

Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
Co-Authored-By: GPT-6 Astra via opencodex <noreply@openai.com>
twoimo and others added 25 commits November 16, 2026 02:13
behavior_diff.sh runs 19 commands that stop before any network, AX,
send, or state write (shell completions for five shells, early
validation errors in the context and local-DB commands, clap argument
errors), each under a fresh empty HOME, and diffs two binaries' stdout,
stderr, and exit codes. Base and this branch are identical across all 19.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
validate_private_regular_file and leftover_queue_has_unknown_send only
forwarded their arguments to the auto_reply_runtime functions of the
same name, so main.rs now imports those directly. readiness_integer was
serde_json::Value::as_i64 under another name; its 20 call sites now call
as_i64 directly. The cfg(test)-only is_valid_auto_reply_transition_event_id
wrapped one call, and its only test now asserts on
parse_auto_reply_transition_event_id instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main.rs carried 16 rustfmt hunks. Formatting it first means the code
moved out in later commits arrives already formatted, so those commits
read as pure moves. Only main.rs is touched (rustfmt skip_children).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 195-line ContextSyncLocal arm of main() becomes
cmd_context_sync_local, and the poll-page validation, bounded retry, and
self-classification deferral helpers it alone uses move with it, along
with their seven tests. The arm body is verbatim (compare-arm), the
helpers are verbatim (compare), and the help dump is unchanged.

The local_source_fence test fixture is pub(crate) because two
worker-gate tests still in main.rs build fences with it. rustfmt also
put commands/mod.rs in order (download before experiment).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uard

require_loco_write moves to commands::send, which owns the LOCO write
commands it gates. require_ax_send and require_allowed_send_chat move to
commands::local_send, which owns the AX send path. Their five tests move
with them. All three gates are verbatim apart from becoming pub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The LocalSend arm carried a 190-line immediately-invoked closure that
takes the AutoReply worker's locks, checks persisted readiness and the
CLI enrollment, and binds the source message. It is now
prepare_local_send, returning a LocalSendSetup that names the two lock
guards held for the send. The closure body is unchanged apart from
taking config and chat_name by reference. main() keeps the let-else
early return, so a failed worker setup still skips --completion-promise
exactly as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
About 4,900 lines of auto-reply machinery and 2,400 lines of its tests
lived in main.rs as free functions sharing constants. They now live in
one package, grouped by what each part owns:

- mod.rs: run_auto_reply, room selectors, context checks, preflight
- process.rs: signals, guardian liveness pipe, child process groups
- runner.rs: Python supervisor and reply-runner executable checks
- llm.rs: reply-model choice and readiness probes (local MLX included)
- state_root.rs: private state dirs, owner lock, enrollment and
  aggregate writes, bounded reads
- queue.rs: stopped-room reply-queue schema attestation
- enrollment.rs: CLI enrollment writer and the validators that read it
- bindings.rs: numeric author bindings and bind: selector attestation
- worker_gate.rs: what the worker proves before local-send mutates
- tests.rs: the 52 tests and fixtures for all of the above

Every item is verbatim apart from visibility (compare reports 205
top-level and 65 test items unchanged). Visibility is pub(super), which is
narrower than the crate-wide reach the items had at the crate root.
Only run_auto_reply and worker_gate::prepare_local_send are pub. The
same 229 binary unit tests pass and the help dump is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ten offline context-* arms of main() (index, repair-self-sends,
search, reply-bundle, style-search, recipient-style, response-time,
reply-search, reply-record, reply-update) become cmd_context_* functions.
compare-arm confirms each body is rustfmt of the original arm. The
eight-flag reply-bundle function carries the same too_many_arguments
allowance run_auto_reply already uses for one-to-one CLI flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s::local

local-chats, local-read, local-poll, local-search, and local-schema
become cmd_local_* functions. Four bodies are rustfmt of the original
arm (compare-arm). cmd_local_poll drops the Ok(()) that would follow its
infinite loop, since that would be unreachable.

local-chats --groups used to return from main() before the
--completion-promise [DONE] line. main() keeps that early return
explicitly, so the output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The AxProbe arm's dump/probe output becomes commands::ax_read::cmd_ax_probe,
beside the other read-only AX command. The body is rustfmt of the
original arm (compare-arm).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eleven context command bodies each spelled out "use --db, or fall back
to openkakao_cli::context::default_db_path()". They now call
commands::context::context_db_path, so the fallback lives in one place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ten tests in main.rs exercised commands::watch, commands::profile, and
commands::members internals rather than the CLI. They now sit in those
modules (profile gets a tests.rs that sees its re-exports through
super::*). The tests are verbatim and the same 229 binary unit test
names pass. Tests for util, loco_helpers, the library error type, and
room_catalog stay in main.rs because those modules are outside this
slice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The object-narrowing check existed four times (contracts, runtime x3,
graph-model), next to second copies of the string and finite-number
guards. runtime.ts and graph-model.ts now import record, text and
finiteNumber from contracts.ts.

JarvisCore and KnowledgeHologram each carried their own spherePoint,
CSS-token colour lookup and geometry disposal walk; both now use
core/scene-helpers.ts. The palette fallbacks stay at the call sites, so
the design-contract test still sees them. CoreRenderState.setSignals
uses load-mapping's clamp01 instead of building its own closure on
every poll.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The table was typed Record<string, number>, so a misspelled stage would
have compiled and silently weighed nothing. Types only: the built
bundle is byte-identical.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Several comments told the reader what the code used to do, with the
date it changed. Git holds that; the comments now state only the
constraint that still applies. The stage-weight comment restated the
table under it and is removed.

attest_app_owned_server's docstring listed attest_unknown, which the
function cannot return.

The built front-end bundle is byte-identical.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Two Python tests read Rust source text at fixed paths. The context split
moved the reference-pack schema and prefix filter out of
src/context/mod.rs, and the main.rs split moved LOCAL_MLX_PROBE_TIMEOUT
into src/commands/auto_reply/llm.rs. Read the whole context module and
the new llm.rs instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gitguardian

gitguardian Bot commented Nov 16, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
40131983 Triggered Generic High Entropy Secret c8a34c3 desktop/src-tauri/src/python_bridge.rs View secret
40131983 Triggered Generic High Entropy Secret 3913dac desktop/src-tauri/src/python_bridge.rs View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@twoimo twoimo closed this Nov 16, 2026
@twoimo twoimo reopened this Nov 16, 2026
@twoimo
twoimo merged commit bf22441 into main Nov 16, 2026
4 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant