Name ffmpeg as the cause when videos are skipped at index time - #399
Merged
Merged
Conversation
On a platform with no bundled ffmpeg binary (win_arm64, musl/Alpine, linux armv7 — imageio-ffmpeg's sdist ships no binary), every video fails to yield a frame and is dropped from the index. The completion notice said only "N files could not be read and were skipped", which sends the user hunting for corrupt files that are in fact fine. Count the videos among bad_files and, when ffmpeg is genuinely absent, say so. A mixed run keeps the generic count and appends the cause, since the images failed for their own reasons. ffmpeg_exe() is guarded behind the video count because it re-probes by spawning `ffmpeg -version` whenever it has no binary to report; a photo-only album must not pay for that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial review of the previous commit turned up three defects: - "1 video that need ffmpeg" — the mixed branch singularised the noun but hard-coded the plural verb. The mixed branch always has count >= 2, so the outer verb is always plural and the bug could only show through the inner one; the videos == 1 case was also the one hole in the new tests. - ffmpeg_exe() was called from the asyncio event loop: both callers of _register_unreadable_files_warning are async, while every pre-existing caller is on a worker thread. It stats whatever $IMAGEIO_FFMPEG_EXE names (accepted unchecked by imageio, so possibly a hung network mount) and can spawn an untimed `ffmpeg -version`. Added ffmpeg_known_unavailable(), which reports the probe every video failure already performed on a worker thread and does no work of its own. - The notice asserted "no ffmpeg binary is available on this system". imageio memoizes its own negative result in an lru_cache, and _is_valid_exe swallows OSError, so a binary that exists but failed to exec once (an AV scanner holding a freshly unpacked ffmpeg.exe) stays unavailable for the process lifetime. Reworded to state what this process observed rather than a claim about the machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`.index-status.completed { color: green !important }` is an author
!important declaration, and those outrank *normal* inline declarations —
so `status.style.color = "#ff9800"` never reached the screen and every
completion-with-a-warning rendered identically to a clean success. That
is the one state where the colour carries the message.
Drive it from a `with-warning` class instead, whose rule out-specifies
the base one (three classes to two; both !important, so only specificity
can break the tie), and clear the inline colour that earlier poll ticks
leave behind.
The two existing Jest assertions were checking `status.style.color`,
which jsdom happily records even when a real engine discards it — they
passed throughout. Re-pinned on the class, plus coverage for a stale
with-warning surviving into a later clean run and for the inline colour
left by the scanning branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On a platform with no bundled ffmpeg binary, every video fails to yield a frame and is dropped from the index entirely — no embedding, no filename row, invisible to the grid, slideshow, search and UMAP map. The completion notice said only:
which sends the user hunting for corrupt files that are in fact fine.
imageio-ffmpeg's sdist ships no binary, so this is the real experience on win_arm64, musl/Alpine and linux armv7 — the install succeeds and only fails at runtime.Change
_register_unreadable_files_warningnow counts how many ofbad_filesare videos and, when ffmpeg was observed unavailable, names the cause:3 videos could not be indexed and were skipped: PhotoMapAI could not find a working ffmpeg.4 files could not be read and were skipped, including 2 videos that need ffmpeg, which PhotoMapAI could not find.1 file could not be read and was skipped.The mixed case deliberately keeps the generic count: those images failed for their own reasons, and blaming ffmpeg for all four would be wrong.
Two details worth the reviewer's attention:
ffmpeg_known_unavailable()(new, invideo.py) rather thanffmpeg_exe(). Both callers of the notice builder areasync defbodies running on the event loop, while every pre-existingffmpeg_exe()caller is on a worker thread.ffmpeg_exe()stats whatever$IMAGEIO_FFMPEG_EXEnames — imageio accepts it unchecked, so it may be a hung network mount — and can spawn an untimedffmpeg -version. The new accessor only reports the probe that every video failure already performed on a worker thread, and does no work of its own. It readsFalsebefore anything has probed, so absence of evidence never invents a cause.imageiomemoizes its own negative in anlru_cacheand its_is_valid_exeswallowsOSError, so a binary that exists but failed to exec once (Windows Defender holding a freshly unpackedffmpeg.exe) stays unavailable for the process lifetime. "There is no ffmpeg on this system" would be a lie to someone who can see it on disk.The check is guarded behind the video count (
videos and ffmpeg_known_unavailable()) so an all-photos album can never inherit "could not find ffmpeg" as the explanation for two corrupt JPEGs. There's a test pinning that short-circuit.Testing
995 passed(pytest),891 passed(Jest),ruff check/npm run lint/npm run format:checkall clean.Nine new tests: the three message shapes plus singular/plural agreement in both branches, the ffmpeg-present control, the short-circuit pin, and three for the new accessor (untouched →
False, follows the probe in both directions, and both offfmpeg_exe'sreturn Nonepaths).The second commit is the result of an adversarial fresh-context review of the first, which caught a
1 video that need ffmpegagreement bug, the event-loop call, and the overclaiming wording.Known gaps, not addressed here
Fixed in the third commit — the colour now comes from a.index-status.completed { color: green !important; }beats the inlinestyle.color, so a completion with a warning renders in success-green.with-warningclass that out-specifies the base rule, since an inline declaration cannot beat an author!importantone. The two Jest assertions covering this were checkingstatus.style.color, which jsdom records even when a real engine discards it, so they passed throughout; they're re-pinned on the class. Note I could not verify the cascade in a real browser here (none installed, and jsdom gets the inline case wrong), so this rests on the specificity rule: both declarations are!important, three classes beat two._completion_warningsis not cleared byset_errororstart_operation, only consumed bycomplete_operation. A run that dies after queueing the notice strands it onto the next completed run of that album. Also pre-existing; the content it strands is just more specific now.🤖 Generated with Claude Code