Read the JSON sidecar for videos generated before InvokeAI 7

docs/user-guide/invokeai-integration.md

@@ -195,17 +195,36 @@ panels together cover both the request and the result.
 Any keyframe or source clip that is also in the current album becomes a
 clickable thumbnail, exactly like an image's reference images.
 
+!!! warning "Existing albums need a full re-index, not an update"
+    Generation metadata is read once, when a file is indexed, and stored in
+    the index — so videos that were already indexed keep whatever they were
+    indexed with, and show no parameters until the album is rebuilt.
+
+    <span class="blue-button-text">Update Index</span> will **not** do it.
+    An update compares the files on disk against the ones in the index and
+    processes only what was added or removed; a file already in the index is
+    never re-read, whatever its modification time. Press the red
+    <span class="red-button-text">Rebuild Index</span> button underneath it
+    instead — see [Rebuilding an index from
+    scratch](albums.md#rebuilding-an-index-from-scratch).
+
+Videos generated before InvokeAI 7 carry no record inside the file, but
+InvokeAI kept one in a JSON sidecar under `outputs/videos/sidecars/`, and
+PhotoMapAI reads that too — so an older clip shows the same panel. The
+sidecar is consulted only when the video itself carries no parameters, which
+is also what happens on the rare occasion InvokeAI 7 could not embed the
+record and fell back to writing one.
+
 !!! note
-    Videos already in an album when you upgrade keep whatever metadata
-    they were indexed with. Press **Update Index** and then, if a video
-    still shows no generation parameters, re-index the album: an update
-    only re-reads files whose modification time has changed.
+    Not every older video has recoverable parameters. A sidecar is written
+    for each generated video, but it records the *workflow* and only
+    sometimes the generation parameters — in one real collection, about a
+    fifth of them carried a usable record. The rest show the still-frame
+    panel alone.
 
 !!! note
-    Only videos generated by InvokeAI 7 or later carry this record.
-    Earlier releases kept it in a separate sidecar file next to the
-    video, which PhotoMapAI does not read. The **Remix** and **Recall**
-    buttons are not offered for videos.
+    The **Remix** and **Recall** buttons are not offered for videos:
+    InvokeAI's recall endpoint does not yet accept video parameters.
 
 ---
 

photomap/backend/invokeai_sidecar.py

@@ -0,0 +1,177 @@
+"""The JSON sidecar InvokeAI wrote beside a video before it embedded metadata.
+
+Releases before InvokeAI 7 kept a generated video's generation record in a
+separate file under ``{outputs}/videos/sidecars/``, mirroring the video's own
+subfolder:
+
+    outputs/videos/general/<uuid>.mp4
+    outputs/videos/sidecars/general/<uuid>.json
+
+    outputs/videos/<uuid>.mp4          # no subfolder
+    outputs/videos/sidecars/<uuid>.json
+
+The file holds the same three strings an InvokeAI 7 MP4 carries as keyed
+metadata — ``invokeai_metadata``, ``invokeai_workflow`` and
+``invokeai_graph`` — each a *stringified* JSON document, or null when that
+generation produced none. In a 662-sidecar sample from a real install, 530
+carried a null record (workflow only) and 132 a real one, so "the file exists
+but has nothing for us" is the common case, not an error.
+
+InvokeAI 7 still writes a sidecar when the metadata remux fails, so this is
+not purely a legacy path.
+
+**Finding the sidecar from the video alone.** PhotoMapAI indexes absolute
+file paths and never learns where an InvokeAI ``outputs`` directory begins,
+so the videos root has to be guessed: each ancestor of the video is tried as
+the root, nearest first, and the video's path relative to it is mirrored
+under ``sidecars/``. The walk is bounded because InvokeAI's own subfolder is
+one level deep in practice (the ``general`` / ``intermediate`` / ``user``
+category) even though its validator permits more.
+
+A false positive would need a directory literally named ``sidecars`` holding
+a JSON file with the video's stem, an ``invokeai_metadata`` key, *and* a
+record that looks like InvokeAI's — all three are required rather than
+assumed, and the last is the same test the drawer routes on.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from collections.abc import Iterator
+from pathlib import Path
+from typing import Any
+
+from .metadata_modules.invokemetadata import looks_like_invoke_metadata
+
+logger = logging.getLogger(__name__)
+
+# The directory InvokeAI puts sidecars in, relative to the videos root.
+SIDECAR_DIRNAME = "sidecars"
+
+# How many ancestors of the video to try as the videos root. Measured
+# against a real install: all 661 sidecars there resolved at depth 1, the
+# category directory, and none at any other depth. Depth 0 covers a videos
+# root with no subfolder at all.
+#
+# Deliberately not deeper. InvokeAI's ``video_subfolder`` is a path and its
+# validator would accept more than one segment, but no caller writes one,
+# and each extra level is a file the reader opens *outside* the album the
+# user configured — at depth 3 that is a path at the filesystem root. Since
+# the headroom buys nothing measurable, it is not worth that reach.
+MAX_SUBFOLDER_DEPTH = 1
+
+# A sidecar holds all three strings, the graph being the large one, and runs
+# to a few hundred KiB. The cap only stops a hostile or corrupt file from
+# being read into memory whole, which JSON parsing requires and the MP4 walk
+# does not. Matched to ``mp4_metadata.MAX_TAG_BYTES`` so the same record is
+# not rejected from one source and accepted from the other.
+MAX_SIDECAR_BYTES = 8 * 1024 * 1024
+
+# The key holding the generation record. Required, so that an unrelated
+# ``sidecars`` directory cannot be mistaken for InvokeAI's.
+METADATA_KEY = "invokeai_metadata"
+
+
+def sidecar_candidates(video_path: Path) -> Iterator[Path]:
+    """Where ``video_path``'s sidecar would be, nearest videos root first.
+
+    Yields at most ``MAX_SUBFOLDER_DEPTH + 1`` paths and stops early at the
+    filesystem root. Nothing is touched on disk here — pass a *resolved*
+    path, or a ``..`` in it will survive into the candidate and let the
+    kernel resolve it back out of the ``sidecars`` directory at open time.
+    :func:`read_sidecar_metadata` resolves before calling this.
+    """
+    filename = video_path.stem + ".json"
+    parent = video_path.parent
+    for depth in range(MAX_SUBFOLDER_DEPTH + 1):
+        try:
+            root = video_path.parents[depth]
+        except IndexError:
+            # Ran out of ancestors before the depth limit.
+            return
+        # "" at depth 0, the category at depth 1, and so on. pathlib drops
+        # the "." that ``relative_to`` returns for the former.
+        subpath = parent.relative_to(root)
+        yield root / SIDECAR_DIRNAME / subpath / filename
+
+
+def read_sidecar_metadata(video_path: Path) -> dict[str, Any]:
+    """The generation record from ``video_path``'s sidecar, or ``{}``.
+
+    Returns ``{}`` when there is no sidecar, when the one found carries a
+    null record (the common case — 530 of 662 in the sample install), or
+    when it is unreadable, oversized, not JSON, or not an InvokeAI sidecar
+    at all. Nothing here raises: this runs once per video while indexing a
+    collection that contains whatever is on the user's disk.
+
+    A candidate that exists but yields nothing usable does not end the
+    search — the next ancestor is still tried.
+    """
+    for candidate in sidecar_candidates(Path(video_path).resolve()):
+        try:
+            if not candidate.is_file():
+                continue
+            if candidate.stat().st_size > MAX_SIDECAR_BYTES:
+                logger.warning("Ignoring oversized sidecar %s", candidate)
+                continue
+            text = candidate.read_text(encoding="utf-8")
+        except OSError as e:
+            logger.debug("Could not read sidecar %s: %s", candidate, e)
+            continue
+        except UnicodeDecodeError as e:
+            # Split from the JSON failure below purely so the log names the
+            # right stage — this never reached the parser.
+            logger.warning("Sidecar %s is not UTF-8 text: %s", candidate, e)
+            continue
+
+        payload = _loads(text, candidate)
+        if not isinstance(payload, dict) or METADATA_KEY not in payload:
+            # Not an InvokeAI sidecar — some other file that happens to sit
+            # where one would.
+            continue
+
+        record = payload[METADATA_KEY]
+        if isinstance(record, str):
+            # The written shape: stringified, as into a PNG chunk or an MP4
+            # tag. Every sidecar in the sample install is this or null.
+            record = _loads(record, candidate)
+        if isinstance(record, dict):
+            if not looks_like_invoke_metadata(record):
+                # The key alone is a weak gate: it would hand back whatever
+                # object some other tool stored under that name. Require the
+                # record to look like InvokeAI's, which is the same test the
+                # drawer routes on — and which all 132 records in the sample
+                # install pass.
+                logger.debug(
+                    "Sidecar %s holds no recognisable InvokeAI record", candidate
+                )
+                continue
+            return record
+        if record is not None:
+            logger.warning(
+                "Sidecar record in %s is a %s, not an object",
+                candidate,
+                type(record).__name__,
+            )
+        # `null` is a real and common shape: a generation that produced a
+        # workflow but no record. Keep looking rather than treating the
+        # file's existence as the answer.
+    return {}
+
+
+def _loads(text: str, source: Path) -> Any:
+    """``json.loads``, returning ``None`` instead of raising.
+
+    ``RecursionError`` is the reason this is a helper rather than one more
+    ``except`` clause: deeply nested JSON raises it, it is a ``RuntimeError``
+    and so is caught by neither of the obvious guards, and ~120 KB of nested
+    brackets is three orders of magnitude under ``MAX_SIDECAR_BYTES``. It
+    escaping here does not merely lose the metadata — ``_load_video`` catches
+    it, returns ``None``, and the video is dropped from the album entirely.
+    """
+    try:
+        return json.loads(text)
+    except (ValueError, RecursionError) as e:
+        logger.warning("Sidecar %s does not hold valid JSON: %s", source, e)
+        return None

photomap/backend/metadata_extraction.py

@@ -14,6 +14,7 @@
 
 from PIL import ExifTags, Image
 
+from .invokeai_sidecar import read_sidecar_metadata
 from .mp4_metadata import INVOKEAI_METADATA_KEY, read_mp4_tags
 
 logger = logging.getLogger(__name__)
@@ -81,22 +82,41 @@ def _json_from_chunk(key: str):
 
     @staticmethod
     def extract_video_metadata(video_path: Path) -> dict[str, Any]:
-        """The InvokeAI generation record embedded in a video, or ``{}``.
+        """The InvokeAI generation record for a video, or ``{}``.
 
         The video counterpart of :meth:`extract_image_metadata`: InvokeAI 7
         writes the same ``invokeai_metadata`` JSON a generated PNG carries as
         a text chunk into a generated MP4 as keyed metadata, so both return
         the same flat record and everything downstream is media-agnostic.
 
+        Two sources, in InvokeAI's own reading order: the MP4's keyed
+        metadata first, then the JSON sidecar. The sidecar covers videos
+        generated before InvokeAI 7 embedded anything, and also the modern
+        case where the embedding remux failed and InvokeAI fell back to
+        writing one. Looking in the file first means an InvokeAI 7 video
+        costs no sidecar lookup at all.
+
+        Precedence is on the *record*, not on the source: a file carrying an
+        empty record falls through to the sidecar, because "the MP4 says
+        nothing" and "the MP4 has no tag" are worth the same and the sidecar
+        may still have something to show.
+
         Only the record is read, not the workflow or graph: PhotoMapAI
-        renders neither, and skipping them keeps the read to the few KiB of
-        the record rather than the few hundred KiB of a graph.
+        renders neither, and skipping them keeps the MP4 read to the few KiB
+        of the record rather than the few hundred KiB of a graph.
 
-        Returns ``{}`` for every failure — a video with no tags, a
-        non-InvokeAI video, an unreadable file, a tag that is not JSON, or
-        JSON that is not an object. The caller indexes the video either way;
-        metadata is never worth failing an index over.
+        Returns ``{}`` for every failure — no tags and no sidecar, a
+        non-InvokeAI video, an unreadable file, a payload that is not JSON,
+        or JSON that is not an object. The caller indexes the video either
+        way; metadata is never worth failing an index over.
         """
+        return MetadataExtractor._embedded_video_metadata(
+            video_path
+        ) or read_sidecar_metadata(video_path)
+
+    @staticmethod
+    def _embedded_video_metadata(video_path: Path) -> dict[str, Any]:
+        """The record in the MP4's own keyed metadata, or ``{}``."""
         try:
             tags = read_mp4_tags(video_path, keys=(INVOKEAI_METADATA_KEY,))
         except OSError as e: