fix(docreader): apply PDF image SMask so embedded figures are not all-black - #2775
Open
649111698 wants to merge 1 commit into
Open
fix(docreader): apply PDF image SMask so embedded figures are not all-black#2775649111698 wants to merge 1 commit into
649111698 wants to merge 1 commit into
Conversation
…-black get_bitmap() decodes only the base image plane and ignores any /SMask (soft transparency mask). Figures exported from plotting tools often store their visible content in the mask while the base RGB plane is black, so extraction produced all-black JPEGs for such images. Decode embedded figures via get_bitmap(render=True), which renders through pdfium's imaging pipeline and carries the mask as an alpha channel, then composite over white (JPEG has no alpha; PDF pages render on white). Falls back to the raw decode when rendering is unavailable. On a production PDF with 7 SMask figures, dark-pixel ratio drops from ~100% to 0.7%-4.5%. Opaque images are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PDFs that embed figures with a soft transparency mask (/\SMask) — a common export format from plotting tools — get their embedded figures extracted as all-black JPEGs. The figure content becomes unreadable in the document chunks, and downstream VLM captions / OCR run on a black rectangle (e.g. captions come back as "black background with three white triangular shapes").
Root cause
_extract_embedded_imagesdecoded each embedded image withPdfObject.get_bitmap(), which returns the raw base image plane and ignores the /\SMask. For masked figures the visible content lives in the mask while the base RGB plane is black, so the raw plane decodes to a solid black rectangle, which is then saved as JPEG as-is.Verified on a production PDF: 7 of 23 embedded images carried an /\SMask; each decoded to ~100% dark pixels via
get_bitmap()while the same object rendered normally in any PDF viewer.Fix
New helper
_decode_embedded_image_pil():get_bitmap(render=True)(FPDFImageObj_GetRenderedBitmap), which renders through pdfium's imaging pipeline and carries the mask as an alpha channel.Opaque images are byte-for-byte unaffected.
Testing
docreader/tests/test_pdf_embedded_images.pywith a hand-built PDF that reproduces the exact pattern (black RGB base plane + /\SMask circle): asserts corners become white, the opaque circle keeps its color, and the extracted JPEG is no longer all-black. Also asserts a plain opaque image is unchanged.