Skip to content

fix(native-sidecar): stop retaining disposed VM filesystem snapshots - #2104

Merged
eersnington merged 1 commit into
mainfrom
fix/vm-dispose-snapshot-leak
Nov 12, 2026
Merged

eersnington merged 1 commit into
mainfrom
fix/vm-dispose-snapshot-leak

Conversation

@eersnington

Copy link
Copy Markdown
Member
  • On dispose, the sidecar snapshots the VM's root filesystem and hands it to the stdio bridge. LocalBridge stored each snapshot in a map keyed by VM id and never removed it.
  • VM ids come from a per-process counter and are never reused, so no snapshot was ever read back. Every disposed VM's files stayed in sidecar memory until the sidecar exited.
  • LocalBridge now discards snapshots. Only test bridges implement load/flush with real storage.
  • Adds a Linux nightly test that disposes VMs on a real sidecar and checks sidecar memory stays flat.
Sidecar, 150 create + 2 MB write + dispose cycles Memory (PSS)
main 33 → 985 MiB, ~6.3 MiB per cycle
this PR 33 → 183 MiB, flat after warm-up

- The stdio sidecar bridge stored every disposed VM's root filesystem snapshot keyed by VM id.
- VM ids are never reused in a sidecar process, so the snapshots were never read and each create/dispose cycle leaked the VM's files (~5.6 MiB per cycle in the repro).
- Add a Linux nightly test that checks sidecar memory stays flat across VM create/write/dispose cycles.
@github-actions

Copy link
Copy Markdown
Contributor

📟 Pip-Boy · b027ec3

📦 Packages · fresh npm install resolve, no lockfile · this PR vs base

🔴 Registry drift: @rivet-dev/agentos-core@latest, @rivet-dev/agentos-posix@latest, @rivet-dev/agentos-runtime-core@latest, @rivet-dev/agentos-sandbox@latest is already over budget for anyone installing it today. This PR didn't cause it.

package (linux-x64-gnu) @latest base this PR Δ vs base packages
@rivet-dev/agentos-core 776.0 MB 776.0 MB 776.0 MB 0 · over 550.0 MB budget 10 (0)
@rivet-dev/agentos-posix 703.7 MB 703.7 MB 703.7 MB 0 · over 305.0 MB budget 7 (0)
@rivet-dev/agentos-runtime-core 703.7 MB 703.7 MB 703.7 MB 0 · over 305.0 MB budget 6 (0)
@rivet-dev/agentos-sandbox 813.6 MB 813.7 MB 813.7 MB 0 · over 600.0 MB budget 85 (0)
`@rivet-dev/agentos-core` details
platform base this PR Δ
linux-x64-gnu 776.0 MB 776.0 MB 0
linux-arm64-gnu 779.3 MB 779.3 MB 0
darwin-arm64 452.2 MB 452.2 MB 0
darwin-x64 453.0 MB 453.0 MB 0
win32-x64 182.3 MB 182.3 MB 0

Largest dependencies

package size via
@rivet-dev/agentos-runtime-sidecar-linux-x64-gnu@0.4.7 593.7 MB @rivet-dev/agentos-core › @rivet-dev/agentos-runtime-sidecar › @rivet-dev/agentos-runtime-sidecar-linux-x64-gnu
@rivet-dev/agentos-runtime-core 103.9 MB @rivet-dev/agentos-core › @rivet-dev/agentos-runtime-core
zod@4.6.7 5.9 MB @rivet-dev/agentos-core › zod
@xterm/headless@6.0.0 1.9 MB @rivet-dev/agentos-core › @xterm/headless
zod-to-json-schema@3.25.2 214.1 KB @rivet-dev/agentos-core › @rivet-dev/agentos-runtime-core › zod-to-json-schema
`@rivet-dev/agentos-posix` details
platform base this PR Δ
linux-x64-gnu 703.7 MB 703.7 MB 0
linux-arm64-gnu 707.1 MB 707.1 MB 0
darwin-arm64 379.9 MB 379.9 MB 0
darwin-x64 380.7 MB 380.7 MB 0
win32-x64 110.0 MB 110.0 MB 0

Largest dependencies

package size via
@rivet-dev/agentos-runtime-sidecar-linux-x64-gnu@0.4.7 593.7 MB @rivet-dev/agentos-posix › @rivet-dev/agentos-runtime-core › @rivet-dev/agentos-runtime-sidecar › @rivet-dev/agentos-runtime-sidecar-linux-x64-gnu
@rivet-dev/agentos-runtime-core 103.9 MB @rivet-dev/agentos-posix › @rivet-dev/agentos-runtime-core
zod@4.6.7 5.9 MB @rivet-dev/agentos-posix › @rivet-dev/agentos-runtime-core › zod
zod-to-json-schema@3.25.2 214.1 KB @rivet-dev/agentos-posix › @rivet-dev/agentos-runtime-core › zod-to-json-schema
@bare-ts/lib@0.4.1 82.2 KB @rivet-dev/agentos-posix › @rivet-dev/agentos-runtime-core › @bare-ts/lib
`@rivet-dev/agentos-runtime-core` details
platform base this PR Δ
linux-x64-gnu 703.7 MB 703.7 MB 0
linux-arm64-gnu 707.1 MB 707.1 MB 0
darwin-arm64 379.9 MB 379.9 MB 0
darwin-x64 380.7 MB 380.7 MB 0
win32-x64 110.0 MB 110.0 MB 0

Largest dependencies

package size via
@rivet-dev/agentos-runtime-sidecar-linux-x64-gnu@0.4.7 593.7 MB @rivet-dev/agentos-runtime-core › @rivet-dev/agentos-runtime-sidecar › @rivet-dev/agentos-runtime-sidecar-linux-x64-gnu
zod@4.6.7 5.9 MB @rivet-dev/agentos-runtime-core › zod
zod-to-json-schema@3.25.2 214.1 KB @rivet-dev/agentos-runtime-core › zod-to-json-schema
@bare-ts/lib@0.4.1 82.2 KB @rivet-dev/agentos-runtime-core › @bare-ts/lib
@rivet-dev/agentos-runtime-sidecar@0.4.7 6.5 KB @rivet-dev/agentos-runtime-core › @rivet-dev/agentos-runtime-sidecar
`@rivet-dev/agentos-sandbox` details
platform base this PR Δ
linux-x64-gnu 813.7 MB 813.7 MB 0
linux-arm64-gnu 818.1 MB 818.1 MB 0
darwin-arm64 483.6 MB 483.6 MB 0
darwin-x64 485.1 MB 485.1 MB 0
win32-x64 226.9 MB 226.9 MB 0

Largest dependencies

package size via
@rivet-dev/agentos-runtime-sidecar-linux-x64-gnu@0.4.7 593.7 MB @rivet-dev/agentos-sandbox › @rivet-dev/agentos-core › @rivet-dev/agentos-runtime-sidecar › @rivet-dev/agentos-runtime-sidecar-linux-x64-gnu
@rivet-dev/agentos-runtime-core 103.9 MB @rivet-dev/agentos-sandbox › @rivet-dev/agentos-core › @rivet-dev/agentos-runtime-core
@rivet-dev/agentos-core 70.3 MB @rivet-dev/agentos-sandbox › @rivet-dev/agentos-core
@sandbox-agent/cli-linux-x64@0.4.2 23.5 MB @rivet-dev/agentos-sandbox › sandbox-agent › @sandbox-agent/cli › @sandbox-agent/cli-linux-x64
zod@4.6.7 5.9 MB @rivet-dev/agentos-sandbox › @rivet-dev/agentos-core › zod

Floating 0.x ranges (resolve above their floor today)

range declared by resolves to subtree
@grpc/proto-loader@^0.8.0 @grpc/grpc-js@1.14.7 0.8.1 6.5 MB
@grpc/proto-loader@^0.7.13 dockerode@4.0.12 0.7.15 6.5 MB
tweetnacl@^0.14.3 bcrypt-pbkdf@1.0.2 0.14.5 170.0 KB
mkdirp-classic@^0.5.2 tar-fs@2.1.5 0.5.3 4.4 KB
All 6 packages · linux-x64-gnu
package (linux-x64-gnu) @latest base this PR Δ vs base packages
@rivet-dev/agentos-core 776.0 MB 776.0 MB 776.0 MB 0 · over 550.0 MB budget 10 (0)
@rivet-dev/agentos-node-pty 7.6 KB 18.0 KB 18.0 KB 0 1 (0)
@rivet-dev/agentos-posix 703.7 MB 703.7 MB 703.7 MB 0 · over 305.0 MB budget 7 (0)
@rivet-dev/agentos-runtime-core 703.7 MB 703.7 MB 703.7 MB 0 · over 305.0 MB budget 6 (0)
@rivet-dev/agentos-sandbox 813.6 MB 813.7 MB 813.7 MB 0 · over 600.0 MB budget 85 (0)
@rivet-dev/agentos-toolchain 130.8 KB 141.2 KB 141.2 KB 0 2 (0)

🧹 Rust dependencies · cargo deny + cargo shear

check new in this PR fixed existing on base
advisories ✅ 0 0 45
licenses ✅ 0 0 0
sources ✅ 0 0 0
bans ✅ 0 0 0
unused ✅ 0 0 16

🦀 Binaries · publish Dockerfiles, --release

Base: main@a714718, the merge base.

artifact npm @latest base this PR Δ vs base
agentos-native-sidecar linux-x64-gnu 593.7 MB 144.0 MB 144.0 MB +1.3 KB (+0.0%)

⏱ Timing

job wall time slowest step
changes 26s checkout · 21s
zizmor 10s checkout · 3s
footprint 1m45s Pack this commit · 34s
hygiene 58s checkout · 21s
native · linux-x64-gnu 14m24s Build with the publish Dockerfile · 14m03s
⏭ self-test — skipped
report 18s Write the comment and run summary · 5s
🔭 Full CI trace for this commit

One OpenTelemetry trace of CI and Pip-Boy for b027ec3: every job and step, with the sizes above attached. Run after CI finishes.

pnpm install --frozen-lockfile --filter 'pipboy...'
pnpm --filter pipboy pipboy trace --sha b027ec31638ae0f721508250ff691a941a996ad4

Add --otlp <url> to send it to any OTLP/HTTP collector.

Agent instruction: From a rivet-dev/agentos checkout, run the commands above, then read the trace file the last command prints. List the slowest jobs and steps and explain where CI time went.

@eersnington
eersnington merged commit 082b80a into main Nov 12, 2026
14 of 15 checks passed
@eersnington
eersnington deleted the fix/vm-dispose-snapshot-leak branch November 12, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant