Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -892,6 +892,59 @@ renders the page and looks at the pixels:
And the marks are flagged Print, for the same reason a widget is: without it a
highlight vanishes the moment somebody prints what they marked up.

## Redacting text: removing it, not covering it

`Doc.Redact` takes the text inside a region **out of the page's content
stream**. ISO 32000-1 §12.5.6.23, on applying a redaction:

> Conforming readers that support redaction annotations shall provide a
> mechanism for applying content removal, and they shall **remove all traces**
> of the specified content. If a portion of an image is contained in a
> redaction region, that portion of the image data **shall be destroyed**;
> clipping or image masks shall not be used to hide that data.

⛔ **A black rectangle is not a redaction.** It is the standard data leak: the
text is still in the content stream, so anyone's copy and paste — or
`pdfops text` — reads it straight back out from under the box. So the test that
decides whether this is a redaction at all is not about how the page looks, it
is that **the redacted string does not appear in the bytes that come out**, and
a drawn rectangle fails it.

The operator that drew the text is overwritten with spaces of its own length,
which does two things at once: the string is gone from the stream, and every
offset after it is still where it was, so the text that stays **does not move**.
`extract.Run.Op` says which bytes to overwrite — deciding which glyphs are
inside a region is work `extract` already does, and a second implementation of
it here would cut the document somewhere other than where the caller was told
the text was.

The region is then painted over, because an area that was emptied and not
marked cannot be told from one that was always blank. That paint is a
*marking*, not the redaction: the text is already gone when it is drawn.

⛔ **And it refuses rather than half-redacts**, naming the page and the reason.
A caller who is told can choose another region or another tool; a caller who is
not told believes the document safe to send. It refuses a region holding:

| what | why |
| --- | --- |
| text inside a Form XObject | the bytes are not the page's, and an XObject is **shared** — cleaning one page would change every page that draws it. `RecolourText` stops at the same wall |
| an image | the spec is explicit that a mask or a clip will not do and the pixels must be destroyed; re-encoding an image with them overwritten is not in this release |
| an annotation | its own `/Contents` can hold the very words being redacted |
| an operator only **part** of whose text is inside the region | blanking it would remove text nobody asked about, and the surviving words cannot simply be re-emitted: the bytes in the stream are font codes, and Unicode does not round-trip back to them through a CID font's map. Keeping the survivors' original bytes and replacing the removed ones with an equivalent kern is the way, and it is next |
| text that **reaches outside** the region | a run is the unit that can be removed, so a producer drawing a whole line with one `Tj` makes that line the run — and redacting one word of it would take the line. That is not a leak, which is why it is tempting to allow; it is the other way a redaction betrays its caller, by removing what they did not ask about and not saying so. A region is allowed to miss a run's own box by up to two points, because a caller mapping a lit rectangle back into page space arrives through screen pixels |

The refusal happens when `Redact` is **called**, not when the document is
written — by the time of `Bytes` the caller has already been promised a
redacted document by everything in between.

A redaction also survives the rest of the verbs. That is worth saying because
the write path once dropped a rewritten content stream whenever it stamped a
page: for a recolouring that loses a colour, but for a redaction it would put
the removed text back and draw the fill over it, so the document would **look
redacted and not be**. There is a test that redacts and watermarks the same
page and then looks for the string.

A document assembled from several files can still be given a form: two sources'
forms cannot be merged — their fields may be named the same — but fields the
caller *asked for* come from neither source and so are in no conflict.
Expand Down
11 changes: 11 additions & 0 deletions doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,11 @@ type Doc struct {
// [Doc.Place]; zero means codec.DefaultMaxPixels. See [Doc.SetMaxPixels].
maxPixels int

// redactErr is set during a write when a page that was to be redacted
// could not be, and makes [Doc.Bytes] fail rather than hand back a
// document that looks redacted. Cleared at the start of every write.
redactErr error

// forms are the page-forms already written during the write in progress,
// so a page drawn many times is embedded once. Made in [Doc.Bytes] and
// dropped with it: a reader.Ref belongs to the writer that issued it, and
Expand Down Expand Up @@ -113,6 +118,12 @@ type Page struct {
// in. A pointer rather than a value with a flag beside it, because black
// is a colour somebody asks for.
recolour *[3]float64

// redactions are the regions of this page whose content is to be REMOVED
// from its stream, and the colour painted over each once it is gone. They
// are checked for what cannot be removed when Redact is called rather than
// when the document is written: see [Doc.Redact].
redactions []Redaction
}

// Open reads a PDF file held in memory, with the empty password.
Expand Down
5 changes: 3 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ require (
github.com/go-ooxml/ooxml v0.7.0
github.com/go-ooxml/opc v0.2.1
github.com/go-pdfkit/document v0.2.0
github.com/go-pdfkit/extract v0.9.0
github.com/go-pdfkit/extract v0.10.0
github.com/go-pdfkit/forms v0.6.0
github.com/go-pdfkit/reader v0.10.0
github.com/go-pdfkit/render v0.69.1
Expand All @@ -22,6 +22,7 @@ require (
github.com/breml/rootcerts v0.3.7 // indirect
github.com/dlclark/regexp2/v2 v2.5.2 // indirect
github.com/dop251/goja v0.0.0-20260926152631-39ec2650adc9 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/evanw/esbuild v0.28.2 // indirect
github.com/go-browserhttp/browserhttp v0.2.0 // indirect
github.com/go-gfx/oksvg v0.2.0 // indirect
Expand All @@ -30,7 +31,7 @@ require (
github.com/go-images/jpeg2000 v0.14.0 // indirect
github.com/go-images/png v0.2.0 // indirect
github.com/go-opentype/fonts v0.14.0 // indirect
github.com/go-opentype/opentype v0.16.1 // indirect
github.com/go-opentype/opentype v0.17.0 // indirect
github.com/go-pdfkit/html2pdf v0.4.1 // indirect
github.com/go-pdfkit/pdffont v0.4.0 // indirect
github.com/go-pdfkit/pdfkit v0.20.0 // indirect
Expand Down
12 changes: 12 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ github.com/dlclark/regexp2/v2 v2.5.2 h1:HAsucWRhsqcDzl6Ua9aR8JwYOTzrZyPrF0/FNxJV
github.com/dlclark/regexp2/v2 v2.5.2/go.mod h1:avUrQvPaLz2DrFNHJF0taWAFFX2C1GMSSoeiqFjcBmU=
github.com/dop251/goja v0.0.0-20260926152631-39ec2650adc9 h1:Ksd3Vil12b/ykT5NjGzd3qNc688cJB9OUXJZZChEktI=
github.com/dop251/goja v0.0.0-20260926152631-39ec2650adc9/go.mod h1:u8yZRUavu+N4EnFFy6J5fVtjE7lEcZ2YyV2GcBXY9c8=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 h1:2tV76y6Q9BB+NEBasnqvs7e49aEBFI8ejC89PSnWH+4=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/evanw/esbuild v0.28.2 h1:A2uETn4jrQTcXaT/shwTDTYBxDjl7fV7nXmUrJxfA2w=
github.com/evanw/esbuild v0.28.2/go.mod h1:D2vIQZqV/vIf/VRHtViaUtViZmG7o+kKmlBfVQuRi48=
github.com/go-browserhttp/browserhttp v0.2.0 h1:3RcjOQVHgqhSOmI5cfsURpexw/vetoDUoLXKzgTCSac=
Expand All @@ -34,10 +37,14 @@ github.com/go-opentype/fonts v0.14.0 h1:WHw7k9gkHT1pu9B2yyvd0FWyQqn/4NKFQoKhvMYt
github.com/go-opentype/fonts v0.14.0/go.mod h1:od2QcmdpKo7QCqo6Jb16y4GULk+h2Hhfq6x3xfvn62Q=
github.com/go-opentype/opentype v0.16.1 h1:Hu2RKinqbd/WVSpRz6VrdKUPZUwPTZ4F9yx2HjZ5oCI=
github.com/go-opentype/opentype v0.16.1/go.mod h1:aSxaPkjDzEnSAYB47s0oaHHa/sDViwW+aPlOz1A3dEg=
github.com/go-opentype/opentype v0.17.0 h1:yDLYLg/nUdfSSzm1Yg8t67racyFsamDEtS1gSR7xk1E=
github.com/go-opentype/opentype v0.17.0/go.mod h1:RujaDgR2h/ErgMkc5y/KYbX8NJtWw3yLRtVxbCfdvFQ=
github.com/go-pdfkit/document v0.2.0 h1:4cdQLl4bcF0CxEw5eeZtkSGVimwgkR8jTMg5TKUEy6s=
github.com/go-pdfkit/document v0.2.0/go.mod h1:Lgxm8Na/Nn2EI8Gmh+lpHhcPLwddOTG32GyL4GLki/c=
github.com/go-pdfkit/extract v0.9.0 h1:gmyIFqZBxBBx6ccpUbF0Sw67jUmAp7q9vQ8mEsTyuFA=
github.com/go-pdfkit/extract v0.9.0/go.mod h1:pUzGzNMx0WTsBOzfZ2XKT5ewqgv8cP/I6WpU4isPsmc=
github.com/go-pdfkit/extract v0.10.0 h1:9dezD5WqsMnQ9amqRaxADrjpf2pbe7TTMFUZwHXCTHU=
github.com/go-pdfkit/extract v0.10.0/go.mod h1:MNOx6Y0g//XSSx0lztuLUPDy4y5yubaNVsxJkVq3F6k=
github.com/go-pdfkit/forms v0.6.0 h1:FJ5oHoIMMdbtVYWsyEEbZ8WFoJtSCdlYrjHE9hTSqxY=
github.com/go-pdfkit/forms v0.6.0/go.mod h1:WZD05vH5x34pL7kOp4lVf1yOFsW7wpffj/ZqnHfsVUs=
github.com/go-pdfkit/html2pdf v0.4.1 h1:zZ1sEoG6c3iJoEX4itG2Lul5z0BtyZ51lutzR4MC96M=
Expand Down Expand Up @@ -70,10 +77,13 @@ github.com/go-widgets/painter v0.16.0 h1:j8FFObisySQChY2JiXS7Di2Mo4cqufIc0PDUnoD
github.com/go-widgets/painter v0.16.0/go.mod h1:3wKWjaM8ZgEhir2HX0UDt8GemePKJ7l/yIjQb/vVSWk=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 h1:FKHo8hFI3A+7w0aUQuYXQ+6EN5stWmeY/AZqtM8xk9k=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8/go.mod h1:K1liHPHnj73Fdn/EKuT8nrFqBihUSKXoLYU0BuatOYo=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.18.7 h1:aUyZsS4kH3QTKurYhAOwAHxllVPnOthb3vPfnF1Ehjw=
github.com/klauspost/compress v1.18.7/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/refraction-networking/utls v1.8.2 h1:j4Q1gJj0xngdeH+Ox/qND11aEfhpgoEvV+S9iJ2IdQo=
github.com/refraction-networking/utls v1.8.2/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/sergeymakinen/go-bmp v1.0.0 h1:SdGTzp9WvCV0A1V0mBeaS7kQAwNLdVJbmHlqNWq0R+M=
Expand All @@ -84,6 +94,7 @@ github.com/srwiley/rasterx v0.0.0-20220730225603-2ab79fcdd4ef h1:Ch6Q+AZUxDBCVqd
github.com/srwiley/rasterx v0.0.0-20220730225603-2ab79fcdd4ef/go.mod h1:nXTWP6+gD5+LUJ8krVhhoeHjvHTutPxMYl5SvkcnJNE=
github.com/tannevaled/gobig2 v0.2.0 h1:yx97/ltljALrIWDFVqHjPUIjAp3bfUUhH8Z8UROvNLc=
github.com/tannevaled/gobig2 v0.2.0/go.mod h1:X0S1H+N35kg6zYgVYRqGMsGj3C35zs+iA2r1MXyPcV4=
github.com/ulikunitz/xz v0.5.8/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yuin/goldmark v1.8.7 h1:XT/tNqCIbabh0ln675w4x3dv04kQBKi8NwYqi9IJaxI=
Expand All @@ -99,5 +110,6 @@ golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
rsc.io/pdf v0.1.1 h1:k1MczvYDUvJBe93bYd7wrZLLUEcLZAuF824/I4e5Xr4=
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=
Loading
Loading