feat: bring desktop Studio UI and local playback to web by richiemcilroy · Pull Request #2312 · CapSoftware/Cap

richiemcilroy · 2026-09-18T10:14:58Z

The Next.js editor page mounts the desktop Solid Studio interface. Browser playback renders locally on the viewer's GPU through Rust/WASM, using WebGPU when presentation is healthy and WebGL2 as the fallback. Native editor workers render exports and are acquired only when Export opens. The two paths share project/timeline mapping, video composition, transition and image shaders, gradients, and blur so saved playback can be compared against exported frames. Web and Chrome-extension recordings preserve synchronized screen and webcam clips, with mic, system audio, and input events stored separately. Web captions require Cap Pro and use the existing AssemblyAI transcription path; desktop captions keep their local models.

This revision brings image segments into local playback. The browser decodes EXIF orientation, premultiplied RGBA, and mipmaps using the native image algorithm, uploads bounded GPU textures, and draws image overlays with the desktop shader. A 20-image fixture verifies that active textures stay resident. It also bounds corrective seeks in the HTML-video fallback when a slow decoder trails playback, while scrubs and discontinuities still seek to the requested frame.

Validation

A Chromium browser-only Studio fixture loads paired clips without worker admission before Export. Its saved image-overlay frame compared with a local native worker frame at mean RGB error 0.62 and 35.89 dB PSNR. That local worker binary predates this commit; exact-commit Linux worker integration is pending.
On a local M4, Chromium WebGPU/WebGL2 and headed WebKit WebGL2 render all 20 image overlays. A simulated half-rate HTML decoder produces zero corrective seeks over 1.2 seconds, down from 22 before this fix. The normal 640×360 paired fixture rendered about 50 frames in 850 ms. These short synthetic probes are not low-end or hosted performance guarantees.
Scoped Biome, Solid TypeScript and production Vite builds, browser replay bundling, WebAssembly-target Cargo formatting/check/build/library Clippy, workflow YAML and actionlint, and an isolated nine-file diff check pass. The prior PR head passed amd64/arm64 editor-worker CI, browser recording reliability, root Biome/typecheck, and the Vercel preview. New exact-commit CI is pending.

Release gates

Studio remains a draft and is off by default behind CAP_WEB_EDITOR_STUDIO_ENABLED=enabled. Local playback still lacks faithful desktop frame/notch, cursor/click/zoom, camera cutout/blur/3D, captions/text/keyboard, masks, and scenes. Native export can render effects that browser playback cannot, so complete 1:1 playback/export parity and every desktop editor feature are not yet delivered. Chrome, Firefox, and Safari authenticated record-to-edit-to-export flows, sustained low-end speed, Safari cold start, memory/battery use, and paired audio synchronization need hosted testing. Machines without WebGPU or WebGL2 currently lack preview. Deployment requires an editor-worker pool and recording-bucket ranged CORS exposing Content-Range; no database migration is required.

Confidence Score: 5/5

The changes since the previous review appear safe to merge, with no new actionable failures identified.

Summary

This revision makes two narrowly scoped follow-up changes:

Allows the Windows target workflow’s informational baseline comparison to handle a source file absent from the PR base while preserving current-source test and Clippy gates.
Extends the Studio replay fixture to accept the same two image-path forms as production and explicitly verifies that imported image previews request the relative project path.

_{Reviews (141) · Last reviewed commit: "fix: skip Windows base comparison for ne..."}

superagent-security

Superagent found 2 security concern(s).

richiemcilroy · 2026-09-18T12:14:34Z

hey @greptileai, please re-review the PR

greptile-apps · 2026-09-18T12:18:13Z

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

Narrative Comment Violates Policy apps/chrome-extension/e2e/overlay-ui.spec.ts:732 ▶

This comment merely restates the status assertion directly below it. The repository requires ordinary test expectations to remain self-explanatory unless a comment preserves non-obvious investigation context. This requirement must be satisfied before merging by removing the comment or making the test structure clearer.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

superagent-security

Superagent found 1 security concern(s).

richiemcilroy · 2026-09-18T13:52:20Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-18T14:24:25Z

hey @greptileai, please re-review the PR

superagent-security

Superagent found 1 security concern(s).

richiemcilroy · 2026-09-18T19:24:51Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-18T19:35:35Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T08:34:04Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T08:38:40Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T09:11:00Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T09:38:00Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T10:02:29Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T10:26:12Z

hey @greptileai, please re-review the PR at d6ff28e. Firefox H.264 color correction now reaches the WebCodecs bitmap path, and the browser/native replay records its source color path.

richiemcilroy · 2026-09-22T10:44:03Z

hey @greptileai, please re-review PR #2312 at commit 5c6a06d. The prior Firefox color-path finding is addressed with an assertion that a WebCodecs frame is actually used when H.264 WebCodecs support is reported; the adaptive Safari blur assertion now scales with preview size. Please assess actionable issues on this exact head.

richiemcilroy · 2026-09-22T10:56:48Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T11:10:40Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T11:57:08Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T12:00:41Z

hey @greptileai, please re-review the PR

richiemcilroy · 2026-09-22T12:13:11Z

hey @greptileai, please re-review the PR

feat: bring desktop editor to web

748bac3

vercel Bot deployed to Preview September 18, 2026 10:16 View deployment

superagent-security Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread .github/workflows/docker-build-editor-worker.yml

Comment thread apps/media-server/Dockerfile.editor Outdated

greptile-apps Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread apps/media-server/src/lib/editor-sessions.ts

fix: isolate editor worker from standalone media server

a6e1dd5

vercel Bot deployed to Preview September 18, 2026 10:32 View deployment

richiemcilroy added 2 commits September 18, 2026 13:12

fix: reclaim unattached web editor sessions

b13e7f3

chore: pin editor worker build inputs

aece09d

vercel Bot deployed to Preview September 18, 2026 12:14 View deployment

fix: align GPUI icon inventory with caption controls

f6ec320

superagent-security Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread apps/media-server/src/lib/editor-signed-assets.ts

vercel Bot deployed to Preview September 18, 2026 13:49 View deployment

fix: normalize web recorder warning quotes

c067e9a

vercel Bot deployed to Preview September 18, 2026 13:53 View deployment

fix: gate macOS-only memory pressure variants

86ec71f

vercel Bot deployed to Preview September 18, 2026 14:25 View deployment

improve: show desktop editor while web sessions prepare

fac71b4

vercel Bot deployed to Preview September 18, 2026 19:14 View deployment

fix: verify staged editor assets before reuse

2943fc6

superagent-security Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread apps/web/lib/editor-worker-routing.ts

vercel Bot deployed to Preview September 18, 2026 19:24 View deployment

fix: require HTTPS for remote editor workers

489510d

vercel Bot deployed to Preview September 18, 2026 19:35 View deployment

fix: restore image overlays in web editor

c6dcac6

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread packages/editor-solid-web/src/browser-video-pool.ts Outdated

Comment thread packages/editor-solid-web/browser-replay/generate-indexed-video.py

fix: bound browser preview frame drift

2e2a526

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread packages/editor-solid-web/src/browser-video-pool.ts Outdated

Comment thread packages/editor-solid-web/browser-replay/replay.cjs Outdated

fix: draw decoded preview before media startup completes

d4912b0

richiemcilroy added 2 commits September 22, 2026 10:09

fix: preserve paired Safari camera and audio captures

484f053

fix: snapshot retained browser GPU frames

e593a57

improve: decode browser Studio frames without repeated seeks

c65dca2

improve: stream local Studio decoding and align Firefox color

8787965

fix: align Firefox WebCodecs color with worker frames

d6ff28e

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread apps/media-server/src/__tests__/routes/editor-solid-ui-replay.ts

test: verify WebCodecs color and scaled blur preview

5c6a06d

test: read Firefox color path from editor iframe

5e0bdcd

feat: share image overlays and stabilize browser playback

59d2197

chore: regenerate browser editor WASM bindings

be0063a

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread packages/editor-solid-web/src/browser-local-canvas.ts

richiemcilroy added 2 commits September 22, 2026 13:12

test: accept relative image paths in worker UI replay

d931655

fix: skip Windows base comparison for new source files

7496864

fix: fetch Windows PR base before comparison

84e528c

Conversation

richiemcilroy commented Sep 18, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Validation

Release gates

Confidence Score: 5/5

Summary

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

richiemcilroy commented Sep 18, 2026

Uh oh!

greptile-apps Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments Outside Diff

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

richiemcilroy commented Sep 18, 2026

Uh oh!

richiemcilroy commented Sep 18, 2026

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

richiemcilroy commented Sep 18, 2026

Uh oh!

richiemcilroy commented Sep 18, 2026

Uh oh!

Uh oh!

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

Uh oh!

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

richiemcilroy commented Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

richiemcilroy commented Sep 18, 2026 •

edited by greptile-apps Bot

Loading

greptile-apps Bot commented Sep 18, 2026 •

edited

Loading