Skip to content

fix(observatory): stop health probe racing WebGL startup into demo mode - #1663

Open
Omriaru wants to merge 1 commit into
ruvnet:mainfrom
Omriaru:fix/observatory-health-probe-race
Open

fix(observatory): stop health probe racing WebGL startup into demo mode#1663
Omriaru wants to merge 1 commit into
ruvnet:mainfrom
Omriaru:fix/observatory-health-probe-race

Conversation

@Omriaru

@Omriaru Omriaru commented Aug 20, 2026

Copy link
Copy Markdown

Summary

On a host without GPU acceleration, the Observatory reports a perfectly healthy sensing server as absent and silently switches to DemoDataGenerator — so the HUD shows fabricated vitals that look real. The server is never at fault; the client's own startup sequence aborts its detection probe.

Root cause

_autoDetectLive() is called synchronously from the Observatory constructor (main.js:134), and its probe uses a wall-clock deadline:

fetch(`${base}/health`, { signal: AbortSignal.timeout(1500) })

The code immediately after that call builds the three.js scene and compiles shaders, blocking the main thread. Where WebGL is software-rendered — a VM, or any host without GPU acceleration — that block lasts for seconds, so the 1500 ms deadline expires before the response can ever be handled.

Wrapping window.fetch on such a host shows what actually happens:

http://127.0.0.1:3000/health   resolved 200   after 5612 ms
main-thread jank:              blockedMs: 6931

The response headers arrive fine — but r.json() is aborted mid-body-read by the already-fired signal. The .catch advances to the next candidate, :8765 is refused, and the app concludes:

[Observatory] No sensing server detected, using demo mode

dataSource latches to 'demo', the badge reads DEMO, and HR/BR sit at -- while the demo generator drives the scene.

Fix

  • Defer the probe with a double requestAnimationFrame so it is issued once the main thread is idle, instead of racing scene construction.
  • Raise the timeout to 10 s, because each frame can still stall ~600 ms under software rendering.

The deferral is the substantive fix; the timeout increase is defence-in-depth.

Verification

Against a live ESP32-S3 CSI node (--source esp32), same build, only this patch changed:

badge HR BR
before DEMO -- --
after LIVE 56 10

After the fix, a clean load auto-detects with no manual intervention:

[Observatory] Sensing server detected at http://127.0.0.1:3000 → ws://127.0.0.1:3000/ws/sensing
[Observatory] WebSocket connected

Vitals track /api/v1/vital-signs. On a GPU-accelerated host the behaviour is unchanged — startup is fast enough that the original deadline was never hit, which is why this only bites some users.

Reproducing

Needs a host where WebGL falls back to software rendering (e.g. a VM). Note that headless Chrome's --virtual-time-budget fakes this bug: it fires the 1500 ms abort prematurely and will make an otherwise-healthy host look affected. I used real-time CDP with Page.addScriptToEvaluateOnNewDocument instead.

Related

🤖 Generated with Claude Code

`_autoDetectLive()` ran synchronously in the Observatory constructor, and its
probe used `AbortSignal.timeout(1500)`. That timer is wall-clock, but the code
immediately after it builds the three.js scene and compiles shaders, which
blocks the main thread. Where WebGL is software-rendered (a VM or any host
without GPU acceleration) that block runs for seconds, so the deadline expires
before the response can be handled.

Instrumenting `window.fetch` on such a host shows the same-origin probe
resolving 200 after 5612ms while the main thread is blocked for 6931ms: the
response headers arrive, but `r.json()` is aborted mid-body-read. The `.catch`
then advances to the next candidate, `:8765` is refused, and the UI concludes
"No sensing server detected" and latches to `DemoDataGenerator` — so a fully
healthy server is reported as absent and the HUD presents fabricated vitals as
if they were real.

Defer the probe until after the first rendered frames so it is issued once the
main thread is idle, and raise the timeout to 10s, since each frame can still
stall for ~600ms under software rendering.

Verified against a live ESP32-S3 CSI node: before, the badge read DEMO with
HR/BR at `--`; after, the same build auto-detects and the badge reads LIVE with
vitals tracking `/api/v1/vital-signs`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant