Read and display InvokeAI 7's video metadata

[lstein]

InvokeAI 7 (commit 70329866b0) embeds a generated video's generation record in the .mp4 itself, as QuickTime keyed metadata under the same three keys a generated PNG carries as text chunks. PhotoMapAI now reads the record at index time and renders it in the metadata drawer beneath the still-frame probe panel.

What's here

• photomap/backend/mp4_metadata.py — a pure-Python moov/udta/meta box walk. The bundled imageio-ffmpeg ships no ffprobe, and walking by box sizes costs a handful of seeks whatever the file's size: mdat, which is all of the video data, is stepped over by its declared size and never read. That matters because this runs once per video over collections that are often on a network mount. Every malformed, truncated or non-MP4 file reads as "no tags".
• Storage — the record goes flat into the per-file metadata dict, beside the reserved photomap_video probe key, exactly as an image's record is stored. One renderer serves both, and every existing .npz rewrite path (delete, batch delete, path update) carries it for free.
• Display — GenerationMetadata5 gains the documented video profile (frames, fps, keyframes, source clip and trim range, the Wan and MiniMax H3 auxiliary models, Ref2VA references) and the view exposes it as display-ready tuples. Keyframes and the source clip join reference_images, so they become clickable thumbnails when they're in the same album.

Three things worth reviewing hardest

1. The metadata_version collision — this fixes InvokeAI 7 images too. PhotoMapAI's discriminator over GenerationMetadata2/3/5 is a synthesised int under that name; InvokeAI 7 now stamps its own record version there as a semver string. Left alone, every image and video that release produces fails the discriminated union and degrades to the formatter's flat scalar table. A non-integer value is now moved aside to invoke_record_version and our tag inferred in its place. Renaming our discriminator would have been the tidier fix, but ~15 existing test payloads pass metadata_version: 3/5 explicitly.

2. /invokeai/recall had no video guard, and this branch is what opened the hole. Only use_ref_image resolved the file path, which is where the is_video 400 lives. A video's record is a perfectly good v5 record, so the endpoint would forward a video generation's prompt, model, seed and cfg to InvokeAI's image recall endpoint. Before videos carried a record the request failed only by accident — the metadata dict held nothing parseable. Now guarded at the path and again at the record.

3. Model is extra="forbid", and ModelIdentifierField has a sixth field. submodel_type is dropped from the record only while it is None, so declaring seven new Model-typed fields converted "unknown extra, parses fine" into "the whole union fails and the drawer loses the prompt, model, LoRAs and reference images too". The new fields use a lenient subclass. vae, upscale_model and qwen3_encoder still carry the same pre-existing hazard — deliberately left alone rather than widening this PR.

Deliberately not done

• Pre-7 videos. Earlier releases kept the record in a JSON sidecar under outputs/videos/sidecars/, which is not read.
• Recall / Remix for videos — blocked upstream, not deferred here. InvokeAI's /api/v1/recall/{queue_id} does not accept video parameters yet; it would have to be adjusted on the InvokeAI side before PhotoMapAI has anything to send. Until then the image payload would land in the wrong tab, which is what the guard above refuses — so that guard is the intended end state for this PR, not a stub.
• Videos already indexed keep the metadata they were indexed with, and an update only re-reads files whose mtime moved, so they need a re-index. Nearly moot in practice — only videos generated after the InvokeAI commit above carry a record at all.

Testing

1083 backend + 890 frontend tests pass; ruff, eslint and prettier clean. The second commit is six defects an adversarial fresh-context review of the first found, each reproduced before being fixed and each with a regression test — including an OverflowError on a >308-digit fps that surfaced as a 500 on /retrieve_image (a blank slide) and, via the now-declared field, reached images as well as videos.

tests/backend/test_media/invoke_video.mp4 is tagged by real ffmpeg the way InvokeAI tags its output, with a prompt containing =, ;, # and a line break so the escaped ffmetadata round trip is covered. The box walk is otherwise tested against hand-built hostile files, since the interesting inputs are the ones no writer produces.

Verified in the running app against real InvokeAI 7 output.

🤖 Generated with Claude Code