Repository navigation
redact: destroy the pixels of a JPEG, as plain samples - #87
Merged
Merged
Conversation
ISO 32000-1 12.5.6.23 says a redaction over a picture must DESTROY that portion
of the image data, and that a clip or a mask will not do. Until now a JPEG in
the region was a refusal, which meant a scanned page — one picture, almost
always DCTDecode — could not be redacted at all.
⛔ IT IS WRITTEN BACK AS PLAIN SAMPLES, NOT AS ANOTHER JPEG. Re-encoding
re-quantises block by block, so the blanked picture would come back with EVERY
block changed: the whole photograph degraded in order to destroy a corner of
it. What this writes instead is exactly what a reader of the original would
have seen, with the region destroyed. The test says so in the only way that
claim can be said: it decodes the original itself and compares the kept samples
BYTE FOR BYTE against that decode. Asserting a colour would be asserting what
an encoder does with it, which is a different subject and a moving one; and
asserting "it still looks about the same" would pass for a picture re-encoded
into mud.
The trade is size — a photograph as Flate is several times the same photograph
as a JPEG — and it is said out loud in the doc comment and the README rather
than discovered. It is paid only by a picture a region actually reaches: one
nobody redacted over keeps its DCTDecode stream byte for byte, and there is a
test for that too.
Two new refusals:
- a CMYK JPEG, rather than a conversion. There is no faithful CMYK to RGB
without the output profile the file does not carry, so converting here
would change every colour on the page to destroy a corner of it. image/jpeg
writes Gray, YCbCr and RGBA and has no way to write a CMYK JPEG, so
decodeJPEG is a seam — without it the refusal would be a claim no test
could reach.
- more than 100 megapixels. A JPEG says how big it is in a handful of bytes
and the decoder then allocates width × height × 3, so a file of thirty-nine
bytes can ask for 1.2 GB.
⛔ THE CEILING IS READ FROM THE HEADER, BEFORE ANYTHING IS DECODED, and the
test proves the ORDER and not only the message. jpegMegapixels is a const, not
a knob a test lowers: the fixture is a real bomb, thirty-nine bytes declaring
20000 by 20000 with not one sample in it, run against the bound callers get —
and it asserts the decoder was never reached, because the refusal reads the
same whether the check runs before the allocation or after it. Moving the check
after the decode makes that test red; it leaves the message untouched.
Mutation-checked both ways: +1 on one channel of the kept samples fails the
byte-for-byte test (row 0, column 0 came back [21 200 90] for [20 200 90]), and
the reordered ceiling fails the bomb test.
FuzzRedactOverAJPEG feeds arbitrary bytes to the new decoder arm with the four
seeds committed in source, including the bomb header: 4.7 million executions in
90 s under GOMEMLIMIT=1GiB, no crash and no runaway.
gofmt clean, go vet clean, 100.0% of statements and 0 uncovered blocks of 3873
(covblocks).
Closes #82.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ISO 32000-1 §12.5.6.23 says a redaction over a picture must destroy that
portion of the image data, and that a clip or a mask will not do. Until now a
JPEG in the region was a refusal — so a scanned page, which is one picture and
almost always
DCTDecode, could not be redacted at all.It is written back as plain samples, not as another JPEG
⛔ Re-encoding re-quantises block by block. The blanked picture would come
back with every block changed: the whole photograph degraded in order to
destroy a corner of it. What this writes instead is exactly what a reader of
the original would have seen, with the region destroyed.
The test says that in the only way the claim can be said: it decodes the
original JPEG itself and compares the kept samples byte for byte against
that decode. Asserting a colour would be asserting what an encoder does with
it, which is a different subject and a moving one; and asserting "it still
looks about the same" would pass for a picture re-encoded into mud.
The trade is size — a photograph as Flate is several times the same photograph
as a JPEG — and it is in the doc comment and the README rather than left to be
discovered. It is paid only by a picture a region actually reaches: one nobody
redacted over keeps its
DCTDecodestream byte for byte, and there is a testfor that too.
Two new refusals
width × height × 3, so thirty-nine bytes can ask for 1.2 GBimage/jpegwrites Gray, YCbCr and RGBA and has no way to write a CMYKJPEG, so
decodeJPEGis a seam — without it that refusal would be a claim notest could reach.
The ceiling is a real bomb, against the real bound
⛔
jpegMegapixelsis a const, not a knob a test lowers. The fixture isthirty-nine bytes declaring 20000 by 20000 with not one sample in it, run
against the bound callers actually get — and it asserts that the decoder was
never reached, because the refusal message reads the same whether the check
runs before the allocation or after it.
⛔ The APP0 segment in that fixture is not decoration:
image/jpegreturns froma header-only read at the frame header only once it has seen a JFIF marker, so
without it the fixture is an unexpected EOF and the test proves the "cannot
read" arm instead of the ceiling. The test checks its own fixture first.
Mutated both ways, because a guard that passes its own mutation is vacuous
+1on one channel of the kept samplesFuzzed
FuzzRedactOverAJPEGfeeds arbitrary bytes to the new decoder arm, with thefour seeds committed in source including the bomb header: 4.7 million
executions in 90 s under
GOMEMLIMIT=1GiB, no crash and no runaway.Gates
gofmt -l .silent,go vet ./...silent, 100.0% of statements and 0uncovered blocks of 3873 (
covblocks).The old refusal table row for
DCTDecodemoved rather than disappeared: whatstays refused is a stream that says
DCTDecodeand holds no JPEG, plusJPXDecodeand any other filter this does not undo.Closes #82.
🤖 Generated with Claude Code