Repository navigation
feat(redact): remove the text in a region, and refuse what it cannot - #68
Conversation
Doc.Redact takes the text inside a region OUT of the page's content stream. Closes #67. ISO 32000-1 §12.5.6.23 requires a conforming reader applying a redaction to "remove all traces of the specified content", and says a mask or a clip will not do. A black rectangle over a paragraph is the standard data leak: the text is still in the stream for any copy and paste — or `pdfops text` — to read back from under the box. So the test that decides whether this is a redaction is not about how the page looks: it is that the redacted string does not appear in the bytes that come out, and a drawn rectangle fails it. The operator that drew the text is overwritten with spaces of its own length, which does two jobs at once: the string is gone from the stream, and every offset after it is unmoved, so the text that stays does not shift. extract.Run.Op (v0.10.0) says which bytes to overwrite — deciding which glyphs are in a region is work extract already does, and a second implementation of it here would cut the document somewhere other than where the caller was told the text was. ⛔ It refuses rather than half-redacts, naming the page and the reason: a caller who is not told believes the document safe to send. It refuses a region holding text inside a Form XObject (shared bytes), an image (the pixels must be destroyed, which is not in this release), an annotation (its /Contents can hold the words), or an operator only part of whose text is inside — blanking that would remove text nobody asked about, and the survivors cannot be re-emitted because the stream holds font codes and Unicode does not round-trip back through a CID map. Two write-path traps, both now controlled by tests: - decorate() took the source's content parts whenever it stamped a page, which threw away a rewritten stream. The file already carried a comment about it costing a recolouring its colour; for a redaction it would have put the text back and drawn the fill over it. Both the plain and the watermarked case fail without the fix. - redactedSpans returning "no spans" on an unreadable page would blank nothing and still paint the fill. It returns the error instead, and Doc.Bytes fails the whole write rather than hand back a document that looks redacted. The fill goes through borrowedExtras rather than being appended to the rewritten bytes, because a page's content may end inside a clipping path — which would clip the fill away without a trace — or with a colour or transform still set. 392 functions at 100.0% by `go tool cover -func`, vet clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run is the unit that can be removed, so a run that only overlapped the region was being blanked entire. A producer that draws a whole line with one Tj makes that line the run — so redacting one word of such a line took the line, and nothing said so. That is not a leak, which is why it was tempting to allow. It is the other way a redaction betrays its caller: by removing what they did not ask about. runInside replaces runTouches in both the refusal and the collection of spans to blank, and the comment on each says the other must agree: a run the check passed over and the collection blanked would remove text nobody asked about, and a run the check allowed and the collection skipped would be left on the page under the fill. The slack is two points, and it is a measurement rather than a fudge: a caller mapping a lit rectangle back into page space arrives through screen pixels and is a fraction of a point out in each direction, more below 1:1. Two points is under half a line of ten-point text, so a run from the line above cannot be swallowed by it. With the test reverted to mere overlap, the new refusal is accepted and a region well inside a run's own box takes the whole run — both fail. It also moved the mixed-operator fixture, which was wrong and passing for the wrong reason: the quad clipped "drop" at x 60..80 instead of covering it, so that subtest was exercising this rule and not the one it names. 393 functions at 100.0%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
One more rule, pushed as A run is the unit that can be removed, so a run that only overlapped the region was being blanked entire — and a producer drawing a whole line with one
The slack is two points — a measurement, not a fudge. A caller mapping a lit rectangle back into page space arrives through screen pixels and is a fraction of a point out each way, more below 1:1; two points is under half a line of ten-point text, so a run from the line above cannot be swallowed. Both halves are tested: a region off by a point and a half still redacts, one well inside the run's own box is refused. It also moved the mixed-operator fixture, which was passing for the wrong reason: its quad clipped |
Closes #67.
What
Doc.Redacttakes the text inside a region out of the page's contentstream, and paints the region over.
⛔ The only test that decides whether this is a redaction
ISO 32000-1 §12.5.6.23, on applying one:
A black rectangle over a paragraph is the standard data leak: the text is still
in the stream, so any copy and paste — or
pdfops text— reads it back fromunder the box. So the test that matters is not about how the page looks, it is
that the redacted string does not appear in the bytes that come out, checked
both in the decoded content stream and through
extract.Text.With the blanking disabled, that test fails and prints the leak itself:
— the fill painted over text that is still there, which is the document this
feature exists to prevent.
How
The operator that drew the text is overwritten with spaces of its own
length. That does two jobs at once: the string is gone from the stream, and
every offset after it is unmoved, so the text that stays does not shift
(
TestWhatWasKeptDidNotMovecompares the kept run's x, y and width before andafter). It is the same technique
recolourContentuses a file away, and for oneof the same reasons.
Which bytes to overwrite comes from
extract.Run.Op(extract#31, v0.10.0). Deciding
which glyphs are inside a region is work
extractalready does — the textmatrix, the font metrics, all of it — and a second implementation here would cut
the document somewhere other than where the caller was told the text was.
⛔ It refuses rather than half-redacts
A caller who is not told believes the document safe to send. So
Redactreturns an error, naming the page and the reason, when the region holds:
RecolourTextstops at the same wall/Contentscan hold the very words being redactedAnd it refuses when
Redactis called, not atBytes— by then the callerhas been promised a redacted document by everything in between.
Text in a form that the region does not reach is fine, and tested: the page is
redacted and the form left alone.
Two write-path traps, both now controlled
decoratethrew away the rewritten stream. It took the source's contentparts whenever it stamped a page. The file already carried a comment about
that costing a recolouring its colour; for a redaction it would put the
removed text back and draw the fill over it — a document that looks redacted.
TestARedactionSurvivesAStampredacts and watermarks the same page and thenlooks for the string. Without the fix both that test and the plain case
fail, because the fill always routes through
decorate.redactedSpansreturning "no spans"when a page cannot be read would blank nothing and still paint the fill. It
returns the error, and
Doc.Bytesfails the whole write rather than handback such a file —
TestARedactionThatCannotBeAppliedFailsTheWritechecksthat no bytes come back with the error, and that the next write recovers.
The fill goes through
borrowedExtrasrather than being appended to therewritten bytes, because a page's content may end inside a clipping path — which
would clip the fill away without a trace — or with a colour or transform still
set.
decoratewraps the page in a save and a restore.A page that cannot be read is also where a recolouring and a redaction part
company: for the first it is a lost colour and the page is left alone, for the
second the write fails. Both are tested through the same seam.
Checks
go vetand the full suite clean; 392 functions at 100.0% bygo tool cover -func, which is the instrument the gate reads.gofmt -lreportsaddfields_test.goandcmd/pdfops/addfields.go, which areunformatted on
mainand not touched here — ops' CI has no gofmt step, sonothing is gated on them, but they are worth someone's commit.
Not in this release
ROoverlays andOverlayTextlayout, redacting inside Form XObjects, imagepixel destruction, and a
pdfops redactverb — each worth its own issue nowthat the text path is proven.
🤖 Generated with Claude Code