Write a linearized file, and extend the checker first so the writer is not its own judge

Linearized reads and checks a linearized file. Nothing in the fleet writes one. This is the last row of BentoPDF's secure group that needs no decoder, no OCR engine and no new format reader — the capability sweep counts it as one of sixteen missing, and the other fifteen each need something the fleet does not have.

⛔ The oracle we already have is WEAKER than the property

linearized.go says so itself, and the sentence is the whole reason to open this before writing code:

⛔ It does NOT validate the hint tables. Their format is F.4 and checking them is a separate, much larger job; a file can pass this and still have hint tables that send a reader to the wrong offset.

It also checks only one of Table F.1's claims against the file — /L against len(b). /O, /E, /T and /H are checked for presence and type, never against what is at those offsets.

So a writer that emitted an ordinary PDF with a correct /L and four invented integers would pass Linearized(). A gate built on it alone would be green for the wrong reason, and the green would be indistinguishable from the real thing. That is the failure this issue exists to prevent, not a risk to note in passing.

Two judges, and they must disagree before they agree

Inside: Linearized(), extended — see below. It is the judgement a writer needs during development, on the bytes it has just produced.
Outside: qpdf --check and qpdf --show-linearization (qpdf 12.4.2 is the reference implementation of this format in practice; it validates the hint tables and reports each violation by name). Same role poppler plays in the render tests: a foreign judge the test prints by.

⛔ The control that makes this honest: an ordinary, non-linearized file produced by the same writer must be REFUSED by both judges. A test suite in which every input passes measures nothing. Before the writer exists, the first commit should be the negative: feed today's writer output to qpdf --check and record that it says the file is not linearized.

What has to be built, in order

Annex F.3.1 lays a linearized file out in eleven parts. The writer already has what the layout needs — Reserve() fixes an object number before anything is written, and Put(ref, o) places it — so this does not need a second writer, it needs a second Finish.

Partition: first-page objects against the rest. Needs the page tree, which is document knowledge, so the entry point takes the first page's object set rather than guessing it.
Layout, the eleven parts in F.3.1's order, with the first-page cross-reference table at the FRONT (part 2) and the main one at the end.
Hint tables (F.4): the page offset hint table (F.4.2) and the shared object hint table (F.4.3) are both mandatory. This is the hard half and most of the work.
The fixpoint: /L, /H, /E, /T are offsets known only after layout. Two passes, with placeholders of FIXED WIDTH in the first, or the second pass moves what the first pass measured.

Extending the checker, which is a change of its own

Linearized()'s contract today is "the front of the file says what Annex F requires and what it says matches the file". Checking /O, /E, /T against the file, and parsing the hint tables, is a larger job and changes what a passing result means. It belongs in its own pull request, before the writer, so that the writer is developed against a judge that was not written to let it through.

What has NOT been read yet

ISO 32000-1:2008 Annex F itself, beyond the passages linearized.go quotes, and qpdf's own account of what --show-linearization validates. Both go before the design is called settled; this issue records the shape and the trap, not a finished plan.

Scope

Reader writes the file. The user-facing verb (pdfops linearize, a ribbon press) is a separate issue in go-pdfkit/ops and go-pdfkit/app, and should not be opened until this lands — a ribbon button for a capability that does not exist is the defect this fleet has filed before.

Prior art: priorart "linearize a PDF: write a linearized (web-optimized) file, Annex F" at 2026-11-12 12:02 CET reported prior art; checked: priorart --repo go-pdfkit/reader on linearization returned 25 tracker entries; read the titles: #37 and #38 are the CHECKER (read and check a file's linearization, and its README), the rest match on the generic word 'write' (the serialiser, string encoding, sparse xref). Nothing writes a linearized file. Also ran priorart --repo go-pdfkit/ops: 71 entries, all matching on 'write', none about linearization.

⛔ The foreign judge this issue leans on is not installed in CI, which I should record here before the writer is built on the assumption that it is.

linearized_test.go opens by saying the positives are qpdf's output and not fixtures written to agree with this reading of Annex F, and that every test skips when qpdf is absent "because a skipped test that looks like a pass is how a checker comes to be believed without ever having been checked". .github/workflows/ci.yml installs qemu-user and nothing else, so on CI every one of those tests skips, including the three added in #41. Only TestLinearizedReadsTheCommittedFixture and the committed-fixture cases actually run there — which is precisely why that test was written, per its own comment.

So the state is honest but weaker than it reads: qpdf agreement is measured on a workstation, by whoever runs it, and nothing in CI would notice if it stopped holding.

That matters more for the writer than for the checker. #40 says the writer must be judged by two judges, one of them qpdf --check; if the second judge is absent from CI, the writer is in practice gated by the inside judge alone — the exact arrangement this issue was opened to avoid.

⛔ And the one apt step there is the pattern that caused go-pdfkit/ops#90:

run: sudo apt-get update && sudo apt-get install -y qemu-user qemu-user-binfmt

That one-liner cannot tell a mirror outage from a wrong package name, which is how an unreachable third-party mirror held a tagged release hostage for two hours on 2026-11-11. go-fleettools/fleettools cmd/aptneed exists for it and is in review as #252.

Both are the same one-line change, once that lands:

- name: Install qpdf, the foreign judge the linearization tests print by
  run: |
    GOBIN=/usr/local/bin go install github.com/go-fleettools/fleettools/cmd/aptneed@latest
    sudo aptneed --need qpdf qpdf

⛔ Worth saying plainly: adding qpdf to CI will make tests run that have never run there, and some may not pass first time. That is the point of adding it, and it should land before the writer rather than after, or the writer's first red will be indistinguishable from the judge's arrival.

linearize: reach every branch, and delete the two that no file could

linearize: the /H bound overflowed on 32-bit, and the check is fuzzed…

Bibliography step for the writer, measured rather than assumed — #41 covers the checker, this is what the second judge actually gives us.

⛔⛔ `qpdf --check` EXITS 0 ON A FILE IT SAYS IS NOT LINEARIZED

Measured on qpdf 12.4.2, with a non-linearized copy of the committed fixture (qpdf in.pdf out.pdf, no --linearize):

$ qpdf --check plain.pdf            ; echo $?
...
File is not linearized
...
0
$ qpdf --show-linearization plain.pdf ; echo $?
plain.pdf is not linearized
0

Both commands report the verdict in their text and exit 0 either way. A test harness that shells out and reads $? would call every file linearized, including one with no linearization at all — the foreign judge would be present, green, and measuring nothing. Same shape as the prwait rule already written in CLAUDE.md: bind the verdict to the SHA, and read conclusion, never the exit status.

So the harness for the writer must match the sentence (File is linearized / is not linearized), and must assert the negative control — an ordinary file produced by the same writer is reported as not linearized — or the judge is unfalsifiable.

`--show-linearization` is a STRUCTURAL oracle, not a yes/no

This is the part that changes the writer's plan. On the committed fixture it prints the whole hint-table content and the part assignment:

Page Offsets Hint Table
min_nobjects: 2            first_page_offset: 677
nbits_delta_nobjects: 1    min_page_length: 194
nbits_delta_page_length: 5 min_content_offset: 0
nbits_delta_content_offset: 0
min_content_length: 194    nbits_delta_content_length: 5
nbits_nshared_objects: 1   nbits_shared_identifier: 2
nbits_shared_numerator: 0  shared_denominator: 4
Page 0: nobjects 3, length 225, content_offset 0, content_length 225, nshared 0
Page 1: nobjects 2, length 194, content_offset 0, content_length 194, nshared 1
        identifier 0: 2, numerator 0: 0

Shared Objects Hint Table
first_shared_obj: 0   first_shared_offset: 0
nshared_first_page: 3 nshared_total: 3
nbits_nobjects: 0     min_group_length: 31
nbits_delta_group_length: 7
Shared Object 0: group length 96 / 1: 98 / 2: 31

Computed Linearization Part Assignments
part 4 (root, document open):  5
part 6 (first page):           7-9
part 7 (other pages, private): 1-2
part 8 (other pages, shared):  none
part 9 (everything else):      3

Two consequences:

The writer can be judged field by field, not pass/fail. Run --show-linearization on qpdf's linearization of a document and on ours, and compare the two readings. A disagreement names the field — min_page_length, nbits_delta_content_offset — instead of saying "not linearized". That is a much sharper oracle than Write a linearized file, and extend the checker first so the writer is not its own judge #40 assumed when it was opened.
The part assignment is printed, so the layout can be checked before a single hint byte is written. part 6 (first page): 7-9 is the F.3.1 partition this fleet has to reproduce, and qpdf will compute it for any input. The first milestone of the writer is therefore "our part assignment equals qpdf's for the same document", which needs no hint tables at all and fails loudly if the object partition is wrong.

What this says about the hint tables

Every field above is bit-packed at a width derived from the data (nbits_delta_* beside each min_*). So the work is not writing a struct: it is computing, per document, the minimum of each quantity and the number of bits needed for the largest delta from it. That is where a writer will be wrong in ways /L and /O cannot show, and it is why #41's checker — which deliberately stops before F.4 — is not sufficient on its own.

Still unread

ISO 32000-1:2008 Annex F itself, beyond the passages linearized.go quotes. The above is qpdf's behaviour, which is the implementation everybody uses but is not the specification; where the two disagree the spec wins and the disagreement is worth its own note.

Measured the first milestone from the comment above — "our F.3.1 partition equals qpdf's for the same document" — with a throwaway probe against both committed fixtures, before writing any of it for real. Two findings, and the second is a constraint I would otherwise have built the wrong thing around.

⛔ `/Parent` makes naive reachability useless: everything reads as shared

First attempt: for each page, walk its object graph; an object reached by one page is private to it, by several is shared.

Measured on testdata/linearized.pdf: 6 of 6 objects shared, both private sets empty. A page's dictionary carries /Parent, which reaches the /Pages node, whose /Kids reach every other page — so every page reaches every object, and the partition collapses.

Not following /Parent gives, for that fixture:

	probe	qpdf
part 6 (first page)	7, 8, 9	7-9
part 7 (other pages, private)	1, 2	1-2
part 8 (other pages, shared)	none	none
part 9 (everything else)	3 (the `/Pages` node)	3

Exact agreement — once part 6 is the union of "private to page 1" and "shared with page 1". Object 9 is used by both pages, and qpdf puts it in part 6 rather than part 8, because the first page must be complete at the front of the file; part 8 is for objects shared among later pages only. That is one sentence of Annex F that a reasonable reading of "shared" gets backwards.

⛔⛔ With object streams the unit of placement is the CONTAINER, not the object

The same rule disagrees on testdata/linearized-objstm.pdf:

	probe	qpdf
part 6	8, 9, 12	8-10
part 7	1, 2	1-2
part 9	—	none

Resolving the disputed numbers:

obj  8: Dict /Type=Page      <- the first page
obj  9: Stream               <- its content
obj 10: Stream /Type=ObjStm  <- an OBJECT STREAM
obj 11: Dict /Type=Pages
obj 12: Array                <- used by both pages

The probe names the logical object (12). qpdf names object 10, the object stream that contains it. A compressed object has no byte offset of its own — it lives inside an ObjStm — so it cannot be placed in a part at all. What gets placed is the container, and it must go in the earliest part any of its members requires.

So the writer's partition cannot be computed over logical objects. It has to be computed over placement units: a top-level object is its own unit, and an object stream is one unit carrying all its members. This also means a linearizing writer that wants a small first page may have to re-pack object streams — qpdf's --object-streams=generate produced a layout where part 9 is empty and everything rides in one ObjStm, which is fine for a 2-page file and would not be for a 200-page one.

Where that leaves the milestone

It stands, with the rule stated correctly:

walk each page's graph without /Parent;
part 6 = everything page 1 reaches, shared or not;
part 7 = reached by exactly one later page; part 8 = by several later pages, never by page 1;
map every object to its placement unit first, and partition those.

Confirmed exactly against qpdf on the traditional-cross-reference fixture. The object-stream fixture is the one that found rule 4, which is the argument for having committed both.

Probe deleted; nothing of it is in the branch.

Third measurement, on the part that will be wrong invisibly: the page offset hint table's header. Throwaway probe, deleted; the branch is untouched.

⛔ I have still not read Annex F's Table F.3. What follows is a hypothesis tested against values qpdf had already printed, which is why it is worth anything — the numbers were not mine to choose.

The header layout, established by measurement

Hint stream of testdata/linearized.pdf (object 6, /H [ 550 127 ], << /Filter /FlateDecode /Length 50 /S 43 >>), decoded to 71 bytes, beginning:

00 00 00 02  00 00 02 26  00 01  00 00 00 c2  00 05
00 00 00 00  00 00  00 00 00 c2  00 05  00 01  00 02
00 00  00 04  80 f8 00 40 ...

Read as alternating big-endian 32-bit value / 16-bit bit-width, then four 16-bit fields:

field	read	qpdf
`min_nobjects`	2	2
`first_page_offset`	550	677
`nbits_delta_nobjects`	1	1
`min_page_length`	194	194
`nbits_delta_page_length`	5	5
`min_content_offset`	0	0
`nbits_delta_content_offset`	0	0
`min_content_length`	194	194
`nbits_delta_content_length`	5	5
`nbits_nshared_objects`	1	1
`nbits_shared_identifier`	2	2
`nbits_shared_numerator`	0	0
`shared_denominator`	4	4

Twelve of thirteen agree exactly, and the header is 36 bytes. The bit-packed per-page entries start there; /S in the stream dictionary is 43, which is where the shared-object section begins — so the page section occupies bytes 36–42 and the shared section 43–70. That makes /S a byte offset into the decoded stream, not an object number.

⛔⛔ The one disagreement is the trap: offsets exclude the hint stream itself

Stored first_page_offset is 550. qpdf reports 677. The difference is 127 — exactly the hint stream's own length from /H [ 550 127 ]. And 677 is where object 7, the first page, actually sits.

So the offsets written into the page offset hint table are computed as if the primary hint stream were not in the file, and a reader adds its length back. That has to be true of the format: the hint stream's length is not known until the offsets it contains have been computed, so the numbers cannot be absolute without a fixpoint.

For the writer this is the difference between working and silently wrong: every offset in the table would be off by the hint stream's length, and nothing in /L, /O, /E or /T would show it — the file would pass CheckLinearized from #41 completely, and a reader asking for the first page would land 127 bytes into the wrong object. It is the exact failure mode #40 was opened to guard against, and it is invisible to every check that exists today.

What this adds to the plan

The header is settled enough to write: 13 fields, 36 bytes, the widths above.
The fixpoint is now explicit and is the thing to design around first, not an implementation detail to discover later.
qpdf --show-linearization decodes these fields, so the writer can be compared field by field — and the first_page_offset row is the one to check first, because it is the one where the stored and reported values legitimately differ.

Still unread: Annex F itself. Where it and this measurement disagree, the specification wins and the disagreement deserves its own note.

linearize: the checks review found missing, and the ones qpdf disagre…

linearize: read the line ending instead of guessing it, and correct a…

linearize: refuse a value this build cannot address, instead of trunc…

linearize: the refusal was written once and tested in one place only

linearize: check the parameters against the document, not only agains…

README: the 32-bit refusal, and a count that three rounds of tests ha…

/E is the one parameter the checker still does not gate, and it is the one a writer is most likely to get wrong — so leaving it is precisely the "writer is its own judge" problem this issue is about. I measured what it actually equals. Writing it down rather than implementing it now, because it is a change the size of #41 and not a rider on one.

What `/E` equals, measured

qpdf --show-linearization testdata/linearized.pdf:

first_page_end: 902
...
Computed Linearization Part Assignments
part 4 (root, document open):  5
part 6 (first page):           7-9
part 7 (other pages, private): 1-2
part 9 (everything else):      3

And the file's front cross-reference section — the one its last startxref points at, at byte 216:

xref
4 6
0000000015 00000 n    <- obj 4, the linearization dictionary itself
0000000501 00000 n    <- obj 5, the catalog (part 4)
0000000550 00000 n    <- obj 6, the hint stream (part 5); /H = [550 127]
0000000677 00000 n    <- obj 7 \
0000000773 00000 n    <- obj 8  } part 6, the first page
0000000871 00000 n    <- obj 9 /
trailer << /Root 5 0 R /Size 10 /Prev 1161 ... >>

The front section lists objects 4–9: by construction, every object that precedes /E. Its highest-offset entry is object 9 at 871, and 871 + n + eolAfter(b, 871+n) == 902 == /E.

So the rule is:

/E is the end of the object with the greatest offset in the first-page cross-reference section.

That is structural, not a coincidence of one file. An earlier reading of it ("obj 9 at 871 gives 902") was true but rested on a single fixture; what makes it a rule is why that section contains those objects.

Both shapes

testdata/linearized-objstm.pdf is the same structure through an XRef stream: object 5 at byte 216, /Index [ 4 9 ], /W [ 1 2 1 ], /Prev 1327, and /E 1131. So the check must read one section in both shapes, and must take the maximum over type-1 entries only — an object inside an object stream has no file offset of its own, but its container does, so the maximum is unaffected.

Why it is not a small change

It needs the entries of one cross-reference section in isolation. readXrefSection (xref.go:110) reads one section, but as a *Document method that merges into the document's table; Open has already walked /Prev by then. Reading the front section alone without disturbing that needs a seam.
eolAfter/spans already encode the end-of-object subtlety (linearize: check the parameters against the document, not only against themselves #41), and this must reuse them rather than re-derive the rule.
The repository gates on exact 100% block coverage, and every new bound needs a mutation that turns a named test red — the spans-subsumption trap from linearize: check the parameters against the document, not only against themselves #41 round 3 applies directly here, since an /E bound could easily be subsumed by the existing "inside the file" check.

What the checker does today

Only 0 < /E <= len(b), documented in the README as explicitly NOT a gate, so a writer that miscomputes /E passes here and fails qpdf. That wording stays honest until the above lands.

Refs #41.

I prototyped the rule from my previous comment and measured it on both shapes — it holds — then found the reason not to ship it yet. Recording both, because the second part is the kind of thing that becomes a latent bug.

The rule verified, both shapes

Reading the front section in isolation (a fresh Document with readXrefSection called once — setEntry keeps the first entry seen, and in a linearized file startxref points at the front section, so its entries are exactly that section's):

linearized.pdf:        /E=902,  /H=[550 127], 6 entries
  objs 4,5,6,7,8,9 at 15,501,550,677,773,871   (all kind 'n')
  => highest non-hint obj 9 at 871, len 30, end 902   == /E  ✅

linearized-objstm.pdf: /E=1131, /H=[630 127], 9 entries
  objs 4,5,6,7,8,9,10 at 15,216,580,630,757,855,953   (kind 'n')
  objs 11,12 kind 'o' inside object stream 10          (no file offset)
  => highest non-hint obj 10 at 953, len 177, end 1131 == /E  ✅

The object-stream shape exercises the part that could have gone wrong: type-2 entries have no offset of their own, their container does, so skipping them leaves the maximum unaffected.

⛔ Why it is not shippable yet: the hint stream's placement

Annex F permits the primary hint stream on either side of the first page. If it follows the first page then /E < hintOffset, the hint stream is still listed in the front section, and a naive "highest offset" would exceed /E and refuse a valid file. So the rule needs the /H objects excluded:

if e.kind != 'n' || int(e.offset) == lin.HintOffset { continue }
if lin.HasOverflowHint && int(e.offset) == lin.OverflowHintOffset { continue }

⛔⛔ And neither committed fixture can tell whether that exclusion works. qpdf places the hint stream before the first page in both — 550 against a maximum of 871, 630 against 953 — so removing the exclusion entirely leaves both green. It is a branch no test can distinguish: present, apparently exercised by the coverage gate (the continue does run), and proving nothing. That is the same vacuity #41 round 3 spent four tests closing, and I would be re-introducing it deliberately.

Shipping the exclusion untested, or dropping it and false-refusing another producer's files, are both bad trades.

What it needs

A fixture whose primary hint stream follows the first page. qpdf does not appear to emit one, so this wants either a hand-built file or another producer — which is worth having on its own terms: both current fixtures come from one tool, so they agree about everything that tool decides, including this.

Until then the README's statement stands and is honest: /E is bounded to 0 < /E <= len(b) and is explicitly NOT a gate.

The seam itself is undramatic and costs nothing to redo: Document is constructible in-package, readXrefSection (xref.go:110) reads exactly one section, and eolAfter/spans already encode the end-of-object rule.

Refs #41.

/E is gated in #43. Recording the next gap while the evidence is in hand, because adding the Ghostscript fixture handed us a ready-made negative case for it.

The hint tables' CONTENTS are still unchecked

The checker verifies that /H points at a real stream and that the stream's extent matches what /H declares. It does not decode the tables or compare what they say with the file. qpdf --show-linearization does, and on the new fixture it disagrees with Ghostscript five times:

WARNING: shared object 0 length mismatch: hint table = 65536; computed = 115
WARNING: shared object 1 length mismatch: hint table = 65536; computed = 118
WARNING: shared object 2 length mismatch: hint table = 65536; computed = 19
WARNING: page length mismatch for page 0: hint table = 230; computed length = 252 (offset = 577)
WARNING: page length mismatch for page 1: hint table = 242; computed length = 248 (offset = 967)

65536 for every shared object is 1 << 16, which looks like a field written at a fixed width rather than computed — three identical wrong values is not three independent mistakes.

⛔ Our checker accepts this file (measured: deep check ACCEPTED), and so it should today — nothing it claims to check is wrong. But it means testdata/linearized-gs.pdf is simultaneously:

the positive fixture that makes the /H exclusion testable (linearize: /E is the end of the first page, and the file already says where that is #43), and
a negative fixture waiting for whoever checks hint-table contents, with qpdf's five specific disagreements as the expected findings.

That is unusually convenient and worth not losing: the hard part of such a check is normally finding a real file that is wrong in a known way, and this one is wrong in five.

⚠ One caution for whoever takes it: a file qpdf warns about is not automatically a file that violates Annex F — qpdf's computation could be the stricter reading. The warnings are a lead, and the spec is the judge. Worth settling before a gate is written on them, not after.

Where this leaves #40

The checker now establishes, against the document: /L, /N, /O, /H's extent, /T (both meanings), and /E. What a writer could still get wrong without being told: the hint tables' contents, and /P, which is a number nothing cross-checks.

Refs #43.

linearize: the /S test was weaker than the /S check, and -1 was a pos…

linearize: the shared object hint table is required, and nothing look…

First of the ten properties from the Annex F reading has landed (#46), and one limit it leaves open is recorded here because the README now points at this issue for it.

Landed

CheckLinearized now requires the primary hint stream's dictionary to carry /S, as a direct integer that is not negative.

⭐ The question it stands or falls on is settled, and by the spec rather than by qpdf's behaviour — §F.4.2:

All objects associated with the first page (part 6) shall have entries in the shared object hint table, regardless of whether they are actually shared.

The first page always has at least its page object, so the table is never empty and no conforming writer may omit /S. Table F.2's flat "(Required)" is not a default that a document without shared objects escapes. Had that gone the other way this check would have refused conforming files, which is the worst thing a checker can do.

Not stricter than the reference either: qpdf --check refuses a /S-less file too, with ERROR: Integer operation attempted on invalid object — the symptom of reading a missing /S as an integer rather than a statement about what is absent.

⚠ Open: the upper bound on `/S`

/S 99 on the committed fixture is accepted here and reported by qpdf as /S (shared object) offset is out of bounds.

The reason it is not closed in #46 is worth stating rather than leaving as an omission: /S is an offset into the decoded stream data. Bounding it means decoding the hint stream, which is the first step of reading the tables themselves — so it belongs with §F.4 and not smuggled into a change about the dictionary. The same applies to "the page offset table shall be the first table and shall start at offset 0": both become checkable the moment the stream is decoded, and neither is checkable before.

So §F.4 now has a concrete first deliverable that is not the whole table format: decode the primary hint stream and bound every position its dictionary names.

Where the ten stand

§	property	state
F.3.6	primary dictionary carries `/S`	✅ #46
F.3.6	hint dictionary entries are direct	◐ `/S` only
F.3.6	positions lie inside the decoded data	⚠ open, needs decode
F.3.6	page offset table first, at offset 0	⚠ open, needs decode
F.3.6	no references to the hint stream anywhere	○
F.3.1	every object has generation 0	○
F.3.1	exactly two cross-reference sections	○
F.3.1	first group numbered after the second	○
F.3.1	lin dict, catalogue, page objects not in an object stream	○
F.3.1	object-stream members take the highest numbers	○
F.3.1	part-11 trailer should not carry `/Prev`	○

⛔ One note on the last row, because it does not belong with the others. It comes from a comment inside Example 11, not from normative text:

trailer
<< /Size 43 >>
% Trailer need not contain other entries; in particular,
% it should not have a Prev entry

"should not", in a comment, in an example — which is weaker than a normative should, let alone a shall. So it is the one row here that must not become a refusal; at most it is something to report. Worth saying before somebody reads this table and implements all eleven alike.

⛔ And a second caveat on the whole table: I derived these from §F.3.1 and §F.3.6. Each still needs its own citation checked at the point of implementation, the way #46's did — the /S row only became safe to enforce once §F.4.2 settled that the shared object table is never empty, and that sentence is nowhere near the row it decides.

Still unread

§F.4's bit-level layouts. The decode step above does not require them; reading the table contents does.

Correcting my own previous comment before anybody acts on it. I wrote that §F.4 "now has a concrete first deliverable that is not the whole table format: decode the primary hint stream and bound every position its dictionary names." That is still the right next step, but it is not the small one it sounds like, and the reason is security rather than effort.

⛔ Bounding `/S` means decoding attacker-controlled bytes in the cheap path

/S is an offset into the decoded stream data, so the bound needs DecodeStream. The hint stream's bytes come straight out of the file, which means an attacker chooses them.

Decoding is capped — filter.go:14, maxDecodedSize = 1 << 30, enforced by readAllCapped through an io.LimitReader, so a decompression bomb is refused rather than fatal. Measured by reading, not assumed.

⚠ But 1 GiB is the cap, and a hint stream is hundreds of bytes to a few kilobytes. A few hundred bytes of Flate in a 1 KB file could make CheckLinearized allocate up to a gigabyte before anything objected — in a function whose entire selling point is that it is the cheap judgement a writer can run on every file it produces. That is a denial-of-service shape, not a correctness one, and it is exactly the sort of thing that gets added without being noticed because the cap already exists and "a bomb is refused" is true.

What that implies for the work, rather than against it

The bound is still worth having; it needs a size limit derived from the file, not the generic 1 GiB one. The hint tables index this document's objects, so their decoded size is a function of the page count and the object count — both known by then. A limit of that shape refuses a hostile stream long before the generic cap and cannot false-refuse a real one.

Deriving it properly needs §F.4's layouts, which is the same reading the table contents need. So the honest ordering is:

read §F.4 (page offset table F.4.2, shared object table F.4.3) ;
derive a decoded-size bound from /N and the object count ;
decode under that bound ;
then bound every position the dictionary names, and check the page offset table starts at 0.

Not: decode first and bound later.

⭐ And it is worth saying plainly that the reviewer who found the /S range gap called it "cheap to add, or document as an open limit". The second half was the right call, and I would not have seen why until I went to add it — the cap that makes decoding safe is also what makes it expensive, and those are easy to confuse.

The README's statement of the limit stands unchanged; this only corrects what closing it costs.

linearize: /S cannot point inside the page offset table's own header

linearize: /S cannot point inside the page offset table's own header (#…

linearize: bound /S above where it costs nothing — an unfiltered hint…

README: /S has a floor and a conditional ceiling, and the judge now a…

Write a linearized file, and extend the checker first so the writer is not its own judge #40

Description

⛔ The oracle we already have is WEAKER than the property

Two judges, and they must disagree before they agree

What has to be built, in order

Extending the checker, which is a change of its own

What has NOT been read yet

Scope

Activity

tannevaled commented on Nov 12, 2026

tannevaled commented on Nov 12, 2026

⛔⛔ qpdf --check EXITS 0 ON A FILE IT SAYS IS NOT LINEARIZED

--show-linearization is a STRUCTURAL oracle, not a yes/no

What this says about the hint tables

Still unread

tannevaled commented on Nov 12, 2026

⛔ /Parent makes naive reachability useless: everything reads as shared

⛔⛔ With object streams the unit of placement is the CONTAINER, not the object

Where that leaves the milestone

tannevaled commented on Nov 12, 2026

The header layout, established by measurement

⛔⛔ The one disagreement is the trap: offsets exclude the hint stream itself

What this adds to the plan

tannevaled commented on Nov 13, 2026

What /E equals, measured

Both shapes

Why it is not a small change

What the checker does today

tannevaled commented on Nov 13, 2026

The rule verified, both shapes

⛔ Why it is not shippable yet: the hint stream's placement

What it needs

tannevaled commented on Nov 13, 2026

The hint tables' CONTENTS are still unchecked

Where this leaves #40

7 remaining items

tannevaled commented on Nov 14, 2026

Landed

⚠ Open: the upper bound on /S

Where the ten stand

Still unread

tannevaled commented on Nov 14, 2026

⛔ Bounding /S means decoding attacker-controlled bytes in the cheap path

What that implies for the work, rather than against it

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

⛔⛔ `qpdf --check` EXITS 0 ON A FILE IT SAYS IS NOT LINEARIZED

`--show-linearization` is a STRUCTURAL oracle, not a yes/no

⛔ `/Parent` makes naive reachability useless: everything reads as shared

What `/E` equals, measured

⚠ Open: the upper bound on `/S`

⛔ Bounding `/S` means decoding attacker-controlled bytes in the cheap path