Stage 8 Phase A: live over event streams — response lifetime, per-scope takeover, chaos knob, room demo - #3653
Conversation
🦋 Changeset detectedLatest commit: 405cfdc The changes in this PR will be included in the next version bump. This PR includes changesets to release 12 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
… to configure Settles the server-components "connection-shaped transport" stage as a framing note rather than a transport: each `live` source keeps its own connection (today's shape) and the response is framed as server-sent events (text/event-stream, no-store, X-Accel-Buffering: no, one codec payload per data: event, comment heartbeat), with Last-Event-ID as position (cursor, or a server-issued digest of the last value) and a terminal record on both framings so death and completion are distinct. The precondition is a backend that holds connections over HTTP/2; dev warns past five live connections over HTTP/1.1. There is no client declaration and no server configuration. RFC 10 gains the `live(fn)` extension surface: lifetime is the response's (nested async sources are branded live and reconnect together), reconnect re-yields the whole answer, SSR hands off at first settlement per hydration scope, undeclared death is an error, hidden pages pause after a grace window with status held through the pause. The transport detour (per-page channel and mailbox, `transport` enum, `hold`, SSE(fn), enableEventStream(), Accept as declaration, framing-follows-method) is recorded under Alternatives considered. §9.5 is rewritten around the same decisions with frames consuming `live` rather than mirroring it; roadmap bullet 8, the §9.3 seed pointer, and the §9.4 grouping seed (which cited a Stage 8 shared connection that no longer exists; see discussion #3603) are corrected. Two shipped-behavior findings are flagged for the implementation: `armLiveTakeover` gates on page-wide hydration end where the snapshot design is per boundary scope (fix: key the gate per scope owner), and `live` will pause on hidden pages. plans/stage8-connection-transport.md carries the phased plan (A: data tier framing, lifetime, takeover fix, pause, chaos knob; B: frames), the public API ledger, open decisions (a)–(f), known costs, and what is future-if-asked-for (`hold`, a socket carrier, built-in sharing, a background opt-out of the pause). Docs only; no changeset. Co-authored-by: Claude via Cursor <cursoragent@cursor.com>
…e, specify the digest skip, park hidden takeovers, close (e) Audit of 247c350 against the tree found the three Public API lists disagreeing on one change and three gaps to close before code: - The terminal record makes a mid-body death reject for EVERY streamed answer, not only live ones (today it is indistinguishable from completion). It was in the plan's ledger but absent from RFC 10's and §9.5's lists; now flagged in all three as a transport-level change. - The value-digest position is specified: computed over the payload string, sent as the event `id:`; a digest-equal reconnect suppresses the first emission only, the iterable does not yield for that connection, later values flow. This is what makes takeover and pause-return free on the wire, and it is a behavior change to shipped `live` (fewer yields) — flagged; the Reconnect bullet no longer says value-shaped sources are unchanged. Plan D12. - A takeover that fires while the page is hidden (opened and hydrated in the background) parks until visible; a consumer ending during a pause fires "closed" and cancels its parked work. Born-hidden and break-during-pause cases added to A2's verify list. - Open (e) closed as the loop's request header (plan D13): "always for streamed answers" contradicted A1's byte-identical verify line, and a server-side `live` declaration has GET's topology caveat so it can cross-check in dev but cannot decide. Notes folded: withdrawn lists aligned across the three files (Accept-as-declaration, `connected`); the multi-line SSE claim corrected (payloads are JSON.stringify output with no raw CR/LF — the split path is defensive, tested with a synthetic payload); the Framing bullet no longer says "nothing else differs"; the D8 fix must keep the per-pass re-arm for islands; §9.5 heading date. Public API now: four flagged behavior changes to shipped `live` (claims nested-async answers; per-scope takeover; hidden-page pause; digest-equal reconnect yields nothing), one to every streamed answer (mid-body death rejects), new wire, dev diagnostics, nothing to configure, no new option. Docs only; no changeset. Co-authored-by: Claude via Cursor <cursoragent@cursor.com>
…live address Two corrections found on entering A1. The terminal record was redundant: seroval already writes a close record for a completed stream and a settlement for a promise, and the decoder's end-of-body sweep (deserializeStream -> deserializeChunk.abort) already rejects every deferred a dying body left open. Verified against the built transport for a top-level generator, a nested stream and a pending promise. F1 described today's behavior, so it is no longer a flagged change; the ledger row and the 'every streamed answer changes' language are removed. D13 re-closed as an address rather than a header: the loop calls <endpoint>/live/<id>, a sibling of /data/<id>, and the server frames what it answers there as an event stream. A third caller kind receiving a third answer shape gets a third path (#3094: caches key on the URL; #3406: reads carry no transport header). Last-Event-ID rides on the live address only. Flagged: live calls move addresses, so a client and server versioned apart miss each other on live calls until both are current. Co-authored-by: Claude via Cursor <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
… (Stage 8 A1) A live loop is a third caller kind receiving a third answer shape, so it gets a third path: <endpoint>/live/<id>, beside /data/<id> (#3094: caches key on the url; #3406: reads carry no transport header). The handler treats it as scripted and frames what it answers there as server-sent events: text/event-stream, Cache-Control: no-store, X-Accel-Buffering: no, the Serialized format tag, one codec payload per data: event, a comment heartbeat every 20s. Thrown and refused outcomes keep the ordinary encoding; the client picks its reader off the content type. Streamed answers at the data address are byte-identical to before. Position: each value-shaped yield carries a digest of its JSON form as the event id (over the value, not the codec record, whose reference ids vary with what preceded them). The loop threads a per-iteration wire slot through its invoke options; the reader records the last id on it and the next connect sends it as Last-Event-ID. A reconnect whose position equals the first yield's digest gets that yield skipped (D12) - fewer yields than before on a digest-equal reconnect. A non-JSON-safe yield carries no id and is never skipped. The reader is built by live() itself so a client that never imports live carries no event-stream parser. Dev builds warn once when a page holds more than five live connections and its document came over HTTP/1.x, naming them and pointing at server.https. New @internal re-exports on the client entry for the frames transport (Phase B): EventStreamReader, createEventChunk, EVENT_STREAM_HEARTBEAT, isEventStream, positionDigest, LAST_EVENT_ID_HEADER. Flagged: live calls move addresses, so a client and server versioned apart miss each other on live calls until both are current. Co-authored-by: Claude via Cursor <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…per-scope takeover The pause bought server cost only - one held connection per source in a background tab, on a backend whose precondition is that it holds connections - for the most intricate state machine on the data tier and a behavior change to shipped live. EventSource and query.live do not pause. The full design (grace window, held status, parked takeover, break during a pause) moves to Future-if-asked-for in the plan; D9 records the deferral; RFC 10 and §9.5 say a hidden page holds its connections. Public API counts go from four live behavior changes to three; the ledger row is removed; A2's verify list and demo lose their pause items. Co-authored-by: Claude via Cursor <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
A generator yields to one reader. Under a server render the serializer
pumping a memo's answer `{ meta, progress: gen }` and a memo reading
`answer().progress` split the generator's yields between them — the
client received half the sequence, the reading memo could miss V1. The
same split met a slot arg the server component also reads.
Every async iterable the runtime reads on the server now goes through a
seat on a shared multicast of the source (`shareAsyncIterable`,
solid-js/server, reached via solid-js/internal): one pump, the whole
sequence for every seat, a log trimmed to the slowest open seat, the
last seat out closes the source. A seat is counted from the moment it
is handed out, so a face preparing a value for the serializer holds its
place before a faster reader can be the last one out.
The serializer takes its seat through the border walk — the frames'
copy-on-write envelope walk, generalized as `toBorderForm(value,
envelopeContainers)`: traced containers → envelopes (frame sink only),
async iterables → seats (every face). `context.serialize` applies it
on the verdict's own `.then` (no added tick) for memo values on the
document face and B3's frame document face; the frame sink applies it
to slot args, and its first-yield tap for document-face async args now
takes two seats instead of handing one cursor between two consumers.
The runtime's own channel wrappers are recognized and passed through.
Projections are untouched: a projection's generator is its own, and
its multi-consumer form is the trace.
Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
A live connection lives as long as its response: an answer holding
nested promises or async iterables keeps the connection open until every
one has settled. The decoder counts what the codec's own close records
leave open (`open()` beside `abort()`); `deserializeStream` hands body
end to the wire slot as `{ open, error, sweep }`; the loop races the
iteration against it — open > 0 at body end is a death (reconnect,
re-yield the whole answer fresh; the sweep is deferred until the
iteration ends for good), open === 0 after top-level done is a
completion. Undeclared answers keep today's behavior.
The server half brands the answer that IS the source — the top-level
iterable, or a function-valued answer (a component). Nothing nested is
branded: sources inside a value answer are bounded and end on their own;
a document render pumps them to their end (their sharing between the
serializer and a reading memo landed in the previous commit).
D8: the hydration takeover gate is keyed per snapshot scope — a live
node in the shell reconnects when the root pass ends, one under a
boundary when that boundary hydrates; a later hydration pass (islands)
arms its own gate. The takeover run stamps the live answer with the
value the page was served with (`solid.LiveResumeFrom`), which the loop
digests into its first connection's `Last-Event-ID`, so a takeover that
finds the same value yields nothing.
Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
`configureServerFunctionsServer({ chaosReconnectEvery: ms })` ends every
live response N ms after it opens, the way a dying connection ends it:
the event-stream writer errors the body with the stream still open, so
the client reads a death (reconnect — backoff, `Last-Event-ID`, the
digest-equal skip, `onstatus`), never a completion. Cleared when the
response completes on its own; inert outside the dev build. Every
event-stream response the live address answers rides the same writer,
so frames' live responses are covered when B2 lands.
Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…time capture) A plain signal created before `hydrate()` began — module-level state minted from `onSettled` during the pass — has no snapshot, so a write to it during capture cascaded through the claim pass (whose DOM writes are skipped) and was lost; a component rendered later in the same pass read a value the server never had. `setSignal` now records the pre-write value as the snapshot at the write and holds it like any other: readers inside the hydrating scope keep the server's value and replay at scope release, readers outside see the write live. Computeds landing async values and projection leaves are excluded, matching creation-time capture. R30 updated. Co-authored-by: Claude via Cursor <noreply@cursor.com> Co-authored-by: Cursor <cursoragent@cursor.com>
The client's `<Loading on>` mounts the dependency node as the owner's first child, so content computes under `id01` and the flatten under `id02`; the server faked a flat boundary (`00`/`01`) and dropped `on` entirely, so every element under a boundary with `on` missed its hydration key — detached duplicates, nothing interactive. The server boundary now passes `on` through and shifts both fake ids when it is present. Co-authored-by: Claude via Cursor <noreply@cursor.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…cted (#3647) `subFetch` primed async-generator computes by pulling their first step under the mocked `fetch`/`Promise`. It now does so only when the value is its own iterator (a generator object). A foreign iterable — a live-call iterable, a deserialized codec stream, the router's `liveQuery` whose `[Symbol.asyncIterator]()` is the subscription itself — was being opened against a `fetch` that never settles (#3647: never connected after hydration) or had its resolver minted through the mocked `Promise` (`temp.s is not a function` on the next streamed value). Fixes #3647 Co-authored-by: Claude via Cursor <noreply@cursor.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…ted value first Two ends of an iteration. A consumer ending it (`return()` — a memo re-invoking with new arguments) completed nothing: every deferred still open in the answer was failed with the transport's own AbortError, and a child memo reading a nested stream halted the page on a route change. The decoder gains `close()` beside `abort()`; `deserializeStream` hands both to the wire slot, and the loop settles what is open by HOW the iteration ended — by error, fail it; by the consumer or by completion, complete the streams and leave promises pending, superseded by the next answer. And the hydration takeover: an iteration stamped with the value the page was served with (`LIVE_RESUME_FROM`) now yields that value locally before it connects. The server's digest-equal skip means the wire may carry no first emission, and the node that re-ran its compute had no other way to land — left pending, it held every write of the tick that released it (the identity minted at `onSettled`) until the source changed. Equality- quiet for a memo, a no-op reconcile for a projection; `Last-Event-ID` and the wire skip are unchanged. Co-authored-by: Claude via Cursor <noreply@cursor.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…test() over an entangled write Harness: `onsettled-write-shell-beside-pending-boundary` (a signal with no snapshot written from `onSettled` during the pass), `loading-on-boundary` and `errored-thunk-fallback-under-loading-on` (element keys under a boundary with `on`). Regenerated artifacts shift only `$R` indices. Signals: a composer's shape — an action call and a clear of its input in one tick. `latest()` shows the entangled clear at once while the committed value holds; a rewrite during the hold shows through `latest()` too and is what lands at settle, not the clear. Co-authored-by: Claude via Cursor <noreply@cursor.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…10 teardown contract
The room: presence, transcript, and directory as `live` sources over a
standing in-memory model, read through memos and a derived optimistic
store; `send` is an action with `until` awaiting the stream's echo, the
composer's clear entangled with it and shown through `latest(text)`.
Status pills off `onstatus`; `<Loading on={room}>` for room switches;
the production harness couples socket close to `request.signal`. Model
lessons that the RFC now carries: a standing source hands the request's
signal to whatever it waits on (or its `finally` runs at its next wake),
a `finally` releases only what its own connection acquired, and a host
owes the request a live `signal`. Plan ledger updated; all Phase A
verify items recorded.
`@solidjs/router` bumped to `2.0.0-next.29` across examples (next.28's
client bundle statically imported the server-functions server entry —
solidjs/solid-router#616, fixed upstream).
Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
7ff3684 to
1544d33
Compare
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Coverage Report for CI Build 36115229959Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage increased (+0.2%) to 73.859%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
Merging this PR will not alter performance
Comparing Footnotes
|
Co-authored-by: Cursor <cursoragent@cursor.com>
Stage 8, Phase A of
documentation/plans/stage8-connection-transport.md:livebecomes one event-stream response per source with nothing to configure, its lifetime is the response's, and a demo exercises the whole contract end to end. RFC 10 (documentation/solid-2.0/10-server-functions.md) is updated in step.What lands
A1 — the live address, event-stream framing. The loop requests
<endpoint>/live/<id>, a sibling of/data/<id>; the server frames whatever it answers there astext/event-stream(no-store,X-Accel-Buffering: no, one codec payload perdata:event, comment heartbeat every 20 s).Last-Event-IDcarries position on reconnect: a cursor source names its own, a value-shaped source gets the server's digest of its last value, and a digest-equal reconnect suppresses only the first emission. Dev warns past five live connections without HTTP/2.A2 — the lifetime is the response's. An answer holding nested promises or async iterables keeps the connection open until each has settled; the decoder counts what the codec's own close records leave open. Body end with deferreds open is a death (reconnect, re-yield the whole answer fresh); with none open, a completion. The server brands only the answer that is the source; nested sources under SSR are shared between the serializer and every reading memo (
shareAsyncIterable). The hydration takeover gate is keyed per snapshot scope, and a takeover iteration yields the value the page was served with locally before it connects (the digest-equal skip means the wire may carry no first emission; left pending, the node held every write of the tick that released it).A3 — dev chaos knob.
configureServerFunctionsServer({ chaosReconnectEvery: ms })ends every live response N ms after it opens the way a dying connection does. Inert outside dev.Fixes found by the demo
live: a consumer ending an iteration withreturn()now completes the nested streams still open and leaves nested promises pending, instead of failing them with the transport's own AbortError (a child memo saw that as an uncaught failure and halted the page on a route change).solid-js: the hydration adoption trace (subFetch) pulls only iterators the compute constructed; a foreign iterable whose[Symbol.asyncIterator]()is the subscription was opened against afetchthat never settles. Fixes Hydrating a server-rendered liveQuery never connects: the hydration trace opens the channel under mocked fetch #3647.solid-js: the serverLoadingdroppedonand faked a flat depth, so every element under a<Loading on>missed its hydration key.@solidjs/signals: a write during hydration capture to a plain signal with no snapshot (module state minted fromonSettled) now records its pre-write value and is held like any other write, instead of cascading through the claim pass and being lost.examples/room— presence, transcript and directory aslivesources over a standing in-memory model;sendis an action withuntilawaiting the stream's echo, the composer's clear entangled with it and shown throughlatest(text); status pills offonstatus;<Loading on={room}>for room switches; a productionserver.jsthat couples socket close torequest.signal. Verified in two headless tabs: chaos →reconnecting → connected (1 reconnect), undeclared summary dies into<Errored>, hidden tab holds, closed tab leaves in ~100 ms.@solidjs/routerbumped to2.0.0-next.29across examples (solidjs/solid-router#616).RFC 10 gains the
livebullets for lifetime, re-yield, bounded nested sources, teardown (a standing source handsrequest.signalto whatever it waits on; afinallyreleases only what its own connection acquired), undeclared death, framing, position, hidden pages; and the host's duty undercreateEvent(request): the request'ssignalmust be live.Public API changes
Per the plan's ledger (
## Public API ledger), each flagged before it landed:<endpoint>/live/<id>. A client and server versioned apart miss each other on live calls until both are current.Last-Event-ID(value digest asid:; cursor sources read the header);X-Accel-Buffering: noandCache-Control: no-storeon live responses.live: nested-async answers keep the connection open (response lifetime); each reconnect re-yields the whole answer fresh.live: a digest-equal reconnect yields nothing — a consumer treating yields as refresh ticks sees fewer of them.live: the takeover fires per snapshot scope, not at page-wide hydration end; the takeover iteration yields the adopted value first, locally.live: ending an iteration byreturn()or completion completes nested streams and leaves nested promises pending; ending by error still fails them.@solidjs/signals:setSignalduring hydration capture snapshots a pre-capture plain signal's pre-write value (held, replayed at scope release).chaosReconnectEveryonconfigureServerFunctionsServer.Loadinghonorson; the adoption trace leaves foreign iterables untouched (Hydrating a server-rendered liveQuery never connects: the hydration trace opens the channel under mocked fetch #3647).@internalexports (not user-facing; ledgered per the fix(web): single-flight delivery is one shared path; frame transform reads consumer-keyed flight data (#3638) #3640 precedent):@solidjs/web/server-functions/client:EVENT_STREAM_HEARTBEAT,EventStreamReader,LAST_EVENT_ID_HEADER,createEventChunk,isEventStream,positionDigest.serverFunctionLiveAddress,LIVE_WIRE,LIVE_RESUME_FROM;parseServerFunctionAddressnow returns{ id, data, live };extractBody/deserializeStreamaccept an optional thirdwireargument.createJSONDeserializerresult gains runtimeopen()/close()besideabort().solid-js/internal:shareAsyncIterable.Withdrawn unbuilt (never shipped):
SSE(fn),enableEventStream(),Accept: text/event-streamas declaration, per-page channel,live: { transport, hold }. The hidden-page pause is designed and deferred; the digest-equal skip already makes a return reconnect free on the wire.Changesets
.changeset/:live-address-event-stream-framing(A1),live-response-lifetime-per-scope-takeover(A2),ssr-shared-async-sources,live-chaos-reconnect-knob(A3),live-return-completes-nested,live-takeover-yields-adopted-value,fix-hydration-trace-foreign-iterable,fix-loading-on-hydration-ids,hydration-write-time-snapshot— allpatch(prerelease).Verification
Full suites green: signals 3513, solid 670, web 852 / 1082 (hydrate) / 196 (server); type tests; prettier. Parity harness scenarios added for write-time snapshot and
<Loading on>.Size
Measured with
scripts/size(brotli, bytes) atnext(c8a9d2369) and at the PR head (1544d332f); caps ratcheted in405cfdc19(chore(size): ledger for Stage 8 Phase A live transport). Attribution is from a per-module esbuild metafile of each scenario bundle at both ends.nextcaptureWriteSnapshotbehindsetSignal(+120 B minified in the core; the only signals change)openScopes/liveGates/nodeGate/takeOver,LIVE_RESUME_FROMstamp,subFetchidentity check) + core +120; web.js 0 BdeserializeStream's live-aware branch (isEventStreamreader choice,connection.endedlifetime signal withopen()/sweep/close). Frames client byte-identical.Dev-only pieces verified folded out of the prod artifacts: the HTTP/1.1 warning (
IS_DEV; absent fromserver-functions/dist/client.jsand the frames bundle) and the chaos knob (server-side, behind the runtimeDEVflag; 0 B in any browser scenario). Theliveloop,EventStreamReader,positionDigest, and the live address are not retained by the frames scenario —live()carries them viawire.open.Worth holding: the frames consumer pays +316 B minified for
deserializeStream's live branch withwirealways undefined there. Theconnection/endclosure could move behind the wire slot the same way the reader did, so a consumer that never importslivecarries none of it.Landing notes
Landed by Claude (via Cursor) from a clean worktree at
c8a9d2369+ this branch. Review findings, recorded so the ledger above is complete:## Public API changesabove (all@internal-tagged, following the existing wire-helper precedent — cf. fix(web): single-flight delivery is one shared path; frame transform reads consumer-keyed flight data (#3638) #3640's "New exports,@internal" entry):@solidjs/web/server-functions/clientgains named exportsEVENT_STREAM_HEARTBEAT,EventStreamReader,LAST_EVENT_ID_HEADER,createEventChunk,isEventStream,positionDigest.shared.d.tsadditionally declaresserverFunctionLiveAddress,LIVE_WIRE,LIVE_RESUME_FROM;parseServerFunctionAddressreturns{ id, data, live };extractBody/deserializeStreamtake an optional thirdwireargument.solid-js/internalgainsshareAsyncIterable(disclosed in thessr-shared-async-sourceschangeset; thesolid-jsstub is@internaland stripped from the public.d.tsbystripInternal).createJSONDeserializer's returned function gains runtimeopen()andclose()besideabort()(untyped, asabortalready was;open()disclosed in the A2 changeset).livesection's "Public API: no new export on the client" is contradicted by the client-entry exports above. The Position bullet says the digest is "computed over the payload string the server already produces";positionDigestdigestsJSON.stringifyof the value, deliberately not the codec payload (its comment explains why: record reference ids depend on prior stream state).snapshot.test.ts: "signal created before capture has no snapshot — writes propagate normally" → held). It is the direct pin of the ledgeredsetSignalbehavior change; no other signals expectation changed.#3647pin covers "not opened under the trace"; the adjacent "shell takeover delivers a first value…" test covers the connection opening after hydration. No test pins that a generator memo's pre-suspension signal read is still tracked through the trace pull (verified by hand at this head: it is — the body runs once under the trace and the memo recomputes after the write). A hand-rolled self-iterator (next+[Symbol.asyncIterator]() { return this }) is still pulled, by design of the identity check.@solidjs/compiler'stsrx-runtime.test.jsfails 8 tests locally with "Runtime bundle still carries build flag(s)__DEV__,__OBSERVE__" — identically atorigin/next, and the rootpnpm testexcludes the compiler package, so it is pre-existing and not this PR's.1a208182d(docs(rfc10): live client exports and position digest match the code): the RFC 10livesection now names the@internalclient exports above and states the position digest is overJSON.stringifyof the emitted value, not the codec payload.