Test to fix
given_large_bodies_when_appending_then_wal_records_should_coalesce_into_one_barrier_group — core/simulator/src/storage/tests.rs:2662, ignored in #4132.
It appends eight prepares whose WAL extents total 32 KiB and asserts they coalesce into one barrier group. They currently take eight.
Why it fails
Mutation::Append.bytes carries record_length(prepare.len()) (core/partitions/src/persistence.rs:683) — the message body. Under segment references the WAL extent is a flat 4096 B (core/journal/src/partition_journal.rs:644-645). The coalescing gate at persistence.rs:1249 compares body bytes against APPEND_BATCH_BYTES_MAX (1 MiB, :38).
Measured against that gate:
| body size |
prepares per sync |
| 1-4 KiB |
64 (as designed) |
| 64 KiB |
15 |
| 512 KiB / 1 MiB |
1 |
At the larger sizes each produce pays a 4096-byte WAL write plus a full barrier set, while the gate believes it just wrote 528 KiB. It also collapses write_segment_bodies' vectored span to a single buffer, undoing that optimisation.
Done when
A separate extent accumulator feeds only the :1249 comparison.
⚠️ Hard constraint. Mutation::Append.bytes, queued_bytes, in_flight_bytes and needs_checkpoint (:934-935) must stay on body bytes. If the extent replaces bytes at in_flight_bytes.set(bytes) (:1269), has_capacity under-counts by up to 64 × (body − 4096), admission over-admits past capacity, and the journal's WouldBlock (partition_journal.rs:646-651) fires inside the worker, where run_inner:1107-1111 converts it into a permanent failure latch — reachable from ordinary produce load.
#[ignore] removed and passing without --ignored.
Found by an adversarial review of #4092. Corresponds to a prior review finding that was marked resolved but was not fixed.
Test to fix
given_large_bodies_when_appending_then_wal_records_should_coalesce_into_one_barrier_group—core/simulator/src/storage/tests.rs:2662, ignored in #4132.It appends eight prepares whose WAL extents total 32 KiB and asserts they coalesce into one barrier group. They currently take eight.
Why it fails
Mutation::Append.bytescarriesrecord_length(prepare.len())(core/partitions/src/persistence.rs:683) — the message body. Under segment references the WAL extent is a flat 4096 B (core/journal/src/partition_journal.rs:644-645). The coalescing gate atpersistence.rs:1249compares body bytes againstAPPEND_BATCH_BYTES_MAX(1 MiB,:38).Measured against that gate:
At the larger sizes each produce pays a 4096-byte WAL write plus a full barrier set, while the gate believes it just wrote 528 KiB. It also collapses
write_segment_bodies' vectored span to a single buffer, undoing that optimisation.Done when
A separate extent accumulator feeds only the
:1249comparison.Mutation::Append.bytes,queued_bytes,in_flight_bytesandneeds_checkpoint(:934-935) must stay on body bytes. If the extent replacesbytesatin_flight_bytes.set(bytes)(:1269),has_capacityunder-counts by up to 64 × (body − 4096), admission over-admits pastcapacity, and the journal'sWouldBlock(partition_journal.rs:646-651) fires inside the worker, whererun_inner:1107-1111converts it into a permanentfailurelatch — reachable from ordinary produce load.#[ignore]removed and passing without--ignored.Found by an adversarial review of #4092. Corresponds to a prior review finding that was marked resolved but was not fixed.