Repository navigation
fix(auto-reply): allow the Flash-Next Sushi model and survive EPERM in the group sweep - #57
Merged
Merged
Conversation
…reply model The owner's local MLX proxy serves beamster/Qwen3.8-Flash-Next-Sushi-2bpw. Startup refuses it today, and the worker falls back to the MLX-Serve Flash-Next ID or sends an mlx/-prefixed form through the unchecked opener. These tests require the Sushi ID, bare or mlx/-prefixed, to pass startup validation, keep its identity through the readiness probe, generate as a Flash-Next text model through the loopback gateway, and reject an answer from another model. Other local IDs, such as the Qwen3.6 35B, stay refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cal Flash-Next model The owner's local MLX proxy serves only the Sushi build of Qwen3.8 Flash-Next. Startup refused it, and the local choice rewrote any Flash-Next request to the MLX-Serve ID, which the proxy does not advertise. Both runtimes now read one table of allowed local models, keyed by exact ID with an optional mlx/ prefix, each with a Flash-Next or 27B family: - Rust: LOCAL_MLX_MODELS in config.rs. AutoReplyLlmChoice::LocalMlx carries the table entry, so apply() and the readiness probe keep the requested ID. The interactive menu lists each local model by ID. - Python: LOCAL_REPLY_MODEL_FAMILIES in auto_reply_ondevice.py. The worker's gateway-model, 27B and gateway-marker checks read it, and the response-model check now catches another model answering a Sushi request. Startup still refuses every other local ID, including the Qwen3.6 35B. The worker's existing rule for mlx/- and omlx/-prefixed IDs, and the menu-bar, Jarvis desktop and model-swap lists, are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eader's group On macOS, killpg can return EPERM for a group whose leader has exited when no member is left that the guardian may signal. In 1.9.0 the session guardian then crashed in _terminate_owned_child with PermissionError: [Errno 1] Operation not permitted. These tests use a fake os.killpg. The guardian must return the child's exit code when every killpg call returns EPERM. The sweep must signal a member that outlives the leader. It must stop without SIGKILL once the group reports EPERM, and survive EPERM from the final SIGKILL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the session guardian's leader exited, _process_group_exists read EPERM from killpg(pgid, 0) as "a member remains". The sweep then sent SIGTERM, which raised PermissionError and crashed the guardian. killpg reports EPERM when no member can be signalled. macOS reports it for a group whose leader has exited. The probe now treats EPERM like ESRCH. A SIGTERM or SIGKILL that fails with EPERM no longer raises. A failed SIGTERM ends the sweep only when the probe confirms no member remains. A member that outlives the leader still gets SIGTERM, then SIGKILL at the end of the grace period, and the sweep stops signalling a group ID once it reports empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The worker picks the first loaded gateway row whose ID contains the requested model's family name, sends the request there, and checks the answer against that substituted ID. These cases pin the requested model instead: - With ddalcu Flash-Next and Sushi both loaded, in either order, each request goes only to the model it names, and an answer from the other one is rejected. - With an unlisted Sushi-4bpw or 27B-8bit row listed first, the Sushi and 27B requests still go to the allowed ID. - When the requested model is not loaded, the request fails closed with mlx_serve_text_model_not_advertised instead of reaching a lookalike. At this commit 12 subtests fail. A Sushi request with only ddalcu loaded returns (0, b'ok', b'') from ddalcu. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The worker chose the gateway row by family name. It took the first loaded row whose ID contained "Qwen3.8-Flash-Next" or "Qwen3.8-27B", sent the request there, and checked the answer against that row. A Sushi request could go to ddalcu Flash-Next or to an unlisted Sushi-4bpw build and pass the answer check. A ddalcu request could likewise go to Sushi. loaded_local_reply_model_id now selects only a loaded row whose canonical ID equals the requested model's canonical ID. If there is no such row, the call fails closed with mlx_serve_text_model_not_advertised, or mlx_serve_vision_model_not_resident for a 27B image request. The answer check compares the response's model with the requested model. The payload still carries the row ID as the gateway lists it. The diagnostic probe and recommend_ondevice_setup still match by family. They report on the gateway and never route a reply. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The owner wants auto-reply running again for one room on the local Qwen3.8 Flash-Next model. Their on-demand MLX proxy serves it as
beamster/Qwen3.8-Flash-Next-Sushi-2bpw. Startup refused that ID. In 1.9.0 the session guardian also crashed withPermissionError: [Errno 1] Operation not permittedwhen it swept the process group of a leader that had already exited.What changed
LOCAL_MLX_MODELSin Rust andLOCAL_REPLY_MODEL_FAMILIESin Python. Each entry has an exact ID, which can take anmlx/prefix, and a Flash-Next or 27B family. The Sushi ID is a Flash-Next entry. The local reply choice carries its table entry, so the requested ID survivesapply()and the readiness probe._terminate_owned_childtreats EPERM fromkillpglike ESRCH, so the guardian returns the child's exit code and does not crash.Scope
Tradeoffs
mlx/form takes the same path as any othermlx/model.Blast Radius
The change touches every local MLX auto-reply startup and generation. A differential check shows the two existing MLX-Serve IDs classify as before in every form. Startup still refuses every other local ID, including the Qwen3.6 35B. The guardian change affects only shutdown after EPERM. A member that outlives the leader still gets SIGTERM, then SIGKILL.
Verification
run_session_guardianintoos.killpg(..., SIGTERM).cargo fmt --checkandcargo clippy --all-targets -- -D warningsare clean.cargo testpasses 1,172, up from 1,169.HOMEunder a write guard, passes 1,372 with 9 skipped, up from 1,366.loaded: trueandstate: "ready". The worker needsowned_by: "mlx-serve"or anmlx/ID.🤖 Generated with Claude Code