Skip to content

Name ffmpeg as the cause when videos are skipped at index time - #399

Merged
lstein merged 3 commits into
masterfrom
lstein/fix/ffmpeg-skip-warning
Sep 20, 2026
Merged

lstein merged 3 commits into
masterfrom
lstein/fix/ffmpeg-skip-warning

Conversation

@lstein

@lstein lstein commented Sep 20, 2026

Copy link
Copy Markdown
Owner

Problem

On a platform with no bundled ffmpeg binary, every video fails to yield a frame and is dropped from the index entirely — no embedding, no filename row, invisible to the grid, slideshow, search and UMAP map. The completion notice said only:

3 files could not be read and were skipped.

which sends the user hunting for corrupt files that are in fact fine. imageio-ffmpeg's sdist ships no binary, so this is the real experience on win_arm64, musl/Alpine and linux armv7 — the install succeeds and only fails at runtime.

Change

_register_unreadable_files_warning now counts how many of bad_files are videos and, when ffmpeg was observed unavailable, names the cause:

case notice
all failures are videos 3 videos could not be indexed and were skipped: PhotoMapAI could not find a working ffmpeg.
mixed failures 4 files could not be read and were skipped, including 2 videos that need ffmpeg, which PhotoMapAI could not find.
ffmpeg present unchanged — 1 file could not be read and was skipped.

The mixed case deliberately keeps the generic count: those images failed for their own reasons, and blaming ffmpeg for all four would be wrong.

Two details worth the reviewer's attention:

  • ffmpeg_known_unavailable() (new, in video.py) rather than ffmpeg_exe(). Both callers of the notice builder are async def bodies running on the event loop, while every pre-existing ffmpeg_exe() caller is on a worker thread. ffmpeg_exe() stats whatever $IMAGEIO_FFMPEG_EXE names — imageio accepts it unchecked, so it may be a hung network mount — and can spawn an untimed ffmpeg -version. The new accessor only reports the probe that every video failure already performed on a worker thread, and does no work of its own. It reads False before anything has probed, so absence of evidence never invents a cause.
  • Worded as what this process observed, not as a claim about the machine. imageio memoizes its own negative in an lru_cache and its _is_valid_exe swallows OSError, so a binary that exists but failed to exec once (Windows Defender holding a freshly unpacked ffmpeg.exe) stays unavailable for the process lifetime. "There is no ffmpeg on this system" would be a lie to someone who can see it on disk.

The check is guarded behind the video count (videos and ffmpeg_known_unavailable()) so an all-photos album can never inherit "could not find ffmpeg" as the explanation for two corrupt JPEGs. There's a test pinning that short-circuit.

Testing

995 passed (pytest), 891 passed (Jest), ruff check / npm run lint / npm run format:check all clean.

Nine new tests: the three message shapes plus singular/plural agreement in both branches, the ffmpeg-present control, the short-circuit pin, and three for the new accessor (untouched → False, follows the probe in both directions, and both of ffmpeg_exe's return None paths).

The second commit is the result of an adversarial fresh-context review of the first, which caught a 1 video that need ffmpeg agreement bug, the event-loop call, and the overclaiming wording.

Known gaps, not addressed here

  • .index-status.completed { color: green !important; } beats the inline style.color, so a completion with a warning renders in success-green. Fixed in the third commit — the colour now comes from a with-warning class that out-specifies the base rule, since an inline declaration cannot beat an author !important one. The two Jest assertions covering this were checking status.style.color, which jsdom records even when a real engine discards it, so they passed throughout; they're re-pinned on the class. Note I could not verify the cascade in a real browser here (none installed, and jsdom gets the inline case wrong), so this rests on the specificity rule: both declarations are !important, three classes beat two.
  • _completion_warnings is not cleared by set_error or start_operation, only consumed by complete_operation. A run that dies after queueing the notice strands it onto the next completed run of that album. Also pre-existing; the content it strands is just more specific now.

🤖 Generated with Claude Code

lstein and others added 3 commits September 20, 2026 09:10
On a platform with no bundled ffmpeg binary (win_arm64, musl/Alpine,
linux armv7 — imageio-ffmpeg's sdist ships no binary), every video fails
to yield a frame and is dropped from the index. The completion notice
said only "N files could not be read and were skipped", which sends the
user hunting for corrupt files that are in fact fine.

Count the videos among bad_files and, when ffmpeg is genuinely absent,
say so. A mixed run keeps the generic count and appends the cause, since
the images failed for their own reasons.

ffmpeg_exe() is guarded behind the video count because it re-probes by
spawning `ffmpeg -version` whenever it has no binary to report; a
photo-only album must not pay for that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial review of the previous commit turned up three defects:

- "1 video that need ffmpeg" — the mixed branch singularised the noun but
  hard-coded the plural verb. The mixed branch always has count >= 2, so
  the outer verb is always plural and the bug could only show through the
  inner one; the videos == 1 case was also the one hole in the new tests.

- ffmpeg_exe() was called from the asyncio event loop: both callers of
  _register_unreadable_files_warning are async, while every pre-existing
  caller is on a worker thread. It stats whatever $IMAGEIO_FFMPEG_EXE
  names (accepted unchecked by imageio, so possibly a hung network mount)
  and can spawn an untimed `ffmpeg -version`. Added
  ffmpeg_known_unavailable(), which reports the probe every video failure
  already performed on a worker thread and does no work of its own.

- The notice asserted "no ffmpeg binary is available on this system".
  imageio memoizes its own negative result in an lru_cache, and
  _is_valid_exe swallows OSError, so a binary that exists but failed to
  exec once (an AV scanner holding a freshly unpacked ffmpeg.exe) stays
  unavailable for the process lifetime. Reworded to state what this
  process observed rather than a claim about the machine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`.index-status.completed { color: green !important }` is an author
!important declaration, and those outrank *normal* inline declarations —
so `status.style.color = "#ff9800"` never reached the screen and every
completion-with-a-warning rendered identically to a clean success. That
is the one state where the colour carries the message.

Drive it from a `with-warning` class instead, whose rule out-specifies
the base one (three classes to two; both !important, so only specificity
can break the tie), and clear the inline colour that earlier poll ticks
leave behind.

The two existing Jest assertions were checking `status.style.color`,
which jsdom happily records even when a real engine discards it — they
passed throughout. Re-pinned on the class, plus coverage for a stale
with-warning surviving into a later clean run and for the inline colour
left by the scanning branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lstein
lstein merged commit a352585 into master Sep 20, 2026
10 checks passed
@lstein
lstein deleted the lstein/fix/ffmpeg-skip-warning branch September 20, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant