Repository navigation
perf(fs): write a writeFiles batch in one sidecar request - #2105
Merged
Merged
Conversation
- Add GuestFilesystemWriteFilesRequest: the sidecar runs mkdir -p and the write for each entry and returns per-entry errors in input order - Bound each request to 16 entries (typed ERR_AGENTOS_RESOURCE_LIMIT) and to the frame limit; the sidecar advertises both in AuthenticatedResponse and the TS and Rust clients split larger batches by those values. A batch runs in one critical section, so the small cap keeps other VMs on the sidecar responsive - Encode generated BARE data fields with serde_bytes so file bytes are copied as one block, not one byte at a time - Skip repeated mkdir -p for a parent already created earlier in the same batch
eersnington
force-pushed
the
perf/write-files-batch
branch
from
November 12, 2026 23:01
0631c48 to
b64c9ea
Compare
Contributor
📟 Pip-Boy ·
|
| job | wall time | slowest step |
|---|---|---|
zizmor |
8s | checkout · 2s |
⊘ changes |
5m01s | checkout · 4m59s |
report |
14s | Write the comment and run summary · 3s |
⏭ footprint |
— | skipped |
⏭ native · ${{ matrix.platform }} |
— | skipped |
⏭ hygiene |
— | skipped |
⏭ self-test |
— | skipped |
🔭 Full CI trace for this commit
One OpenTelemetry trace of CI and Pip-Boy for b64c9ea: every job and step, with the sizes above attached. Run after CI finishes.
pnpm install --frozen-lockfile --filter 'pipboy...'
pnpm --filter pipboy pipboy trace --sha b64c9eab0f97c01b17ea3cd644556b75033830ceAdd --otlp <url> to send it to any OTLP/HTTP collector.
Agent instruction: From a rivet-dev/agentos checkout, run the commands above, then read the trace file the last command prints. List the slowest jobs and steps and explain where CI time went.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
vm.filesystem.writeFiles()sends a batch to the sidecar as oneGuestFilesystemWriteFilesRequestinstead of two round trips (mkdir -p+ write) per file. Per-entry results keep their input order, and a failed entry doesn't stop the rest. TS and Rust clients behave the same.AuthenticatedResponse(maxFrameBytes, newmaxWriteFilesEntries= 16). Clients split larger batches by those values; no client copies a sidecar constant.datafields now decode as one block copy (serde_bytes) instead of byte by byte. Wire bytes are unchanged. This speeds up every binary field in the protocol: one 2.3 MB file write went from 42 to 11 ms.mkdir -pfor parents it already created in the same batch.A batch runs in one sidecar critical section, so the entry cap is kept small. Another VM on the same sidecar waits at most 17–51 ms while 1k–8k files are written (6 s with a 4,096 cap).