Conversation
Declares the interface a host page may expose on window.__TANGLE_PIPELINE_STORAGE_HOST__ and the detection that reads it. Nothing consumes either yet. Detection is deliberately fail-open: a missing global, a wrong shape, a throwing getter or a contract version we do not understand all resolve to undefined, which leaves local storage as the only storage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…it wrote A driver's write() now returns a PipelineFileDescriptor instead of void, so a store that mints its own ids, names and content versions can say so rather than having the registry invent them. addFile adopts the reported externalId when there is one and mints a UUID otherwise, and listing prefers the same field so repeated listings resolve to stable ids. addFile also writes before it registers: the previous order left a registry row pointing at a file that was never created whenever the write was rejected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… its key PipelineFile gains a displayName that falls back to the storage key, so every store whose key is the name reads exactly as before while a store with opaque keys can supply something a person recognises. The three read sites — the folder table's rows, its filter, and the delete confirmation — switch to it; the folder table still navigates by key. useLoadSpec stops assuming the route param equals the storage key. It matched change events against ref.name, which silently stopped reloading the editor once the two could differ, so it now compares against the key of the file the query actually loaded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ore saving A saved pipeline currently carries whatever the in-memory model has accumulated, including per-viewer component-library state (favorited, owned, published_by, superseded_by) and stale input values. Those travel with the file and become someone else's view of their own library. toPortablePipelineSpec reduces a spec to an allowlist of pipeline- describing fields, recursively through subgraphs, and is wired into the two pipeline save paths. dynamicData arguments pass through verbatim, since their shape is not modelled here and filtering inside them would only fail at run time. Allowed keys are copied in source order, so a document that loses nothing serialises byte-for-byte as it did before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
HostStorageDriver adapts the detected host to PipelineStorageDriver. It owns the YAML boundary in both directions: the host exchanges pipeline documents as structured data, so the driver parses and projects on write and re-serialises on read, and refuses a document it cannot recognise. Errors are duck-typed off the rejection value, because instanceof does not survive a window boundary, and every unrecognised code degrades to "unavailable". Each code maps to its own message built from the label the host supplies, so nothing about the store is named here. A save that fails now leaves the editor's autosave indicator showing the reason instead of a cloud tick. When a host is present, a folder pointing at it is seeded once, and its name follows the host's label. When no host is present none of this is reachable: createDriver refuses to build the driver and no folder appears. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…igrate lazily A store that owns its own listing means a file missing from a listing is gone, not merely unlisted, so its registry row is dropped and a moved content version is announced as a remote change so an open editor reloads it. Our own writes record the version they produced, which is what stops a save from reading back as someone else's change. Existing local pipelines migrate one at a time: the first save of a pipeline while a host is present also writes it to the host and remembers the key. The local copy stays authoritative and is never removed. Deleting a pipeline from the host forgets that key everywhere, because writing to a deleted key can revive the dead record and the pipeline would come back wearing its identity. The registry needs an index to find rows by their host key, hence version(2). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🎩 PreviewA preview build has been created at: |
…hout a host Seeding the folder was only half the job. The folder outlives the page that installed it, so any later load without a host left a row whose driver cannot be built — and one such row takes the whole folder listing down with it, not just itself, because listSubfolders maps every entry through createDriver. The pipelines page collapsed to the generic retry icon after a few silent retries. Syncing now runs in both directions, and the registry rows that only ever described the host's files go with the folder. A local pipeline's record of what it already copied to the host survives, so nothing is copied twice when the host comes back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sitting a host-provided store beside browser storage as an extra folder does not hold up. The copy taken on first save is frozen at that moment while the local original keeps being edited, and nothing tells anyone the two have drifted. Editing was the point, so the copy is the wrong artefact. Storage mode is now one decision taken as the app boots: with a host on the page the root folder *is* the host, and browser storage is not a sibling, a fallback, or a cache. The decision holds on to the host it found rather than re-reading the global, because detection is fail-open and a host that later disappears would otherwise read as "no host" and send the next write to the browser — the one outcome this exists to stop. The contract has no folders, so the host root is flat: it lists no subfolders, refuses to create one, and refuses to open a folder id left over from browser storage rather than quietly resolving it locally. Seeding the registry from the legacy list is skipped for the same reason — those names are keys the host has never heard of. The registry's host-key column, the host folder row and the copy-on-save path all go, and version(3) clears out what earlier page loads left behind: a folder row whose driver cannot be built takes the whole listing down. `utils/URL.ts` reaches for `RUNS_BASE_PATH` through the router rather than the route constants it actually lives in, which drags the entire route tree into a leaf util and makes anything importing it uncycleable in tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three call sites each turned a `PipelineRef` into a file their own way, and none of them agreed: one fell back from a missing id to the name and opened whatever happened to share it, one resolved the file a second time inside a floating promise, and one gave up silently. With opaque storage keys the name is a cosmetic slug, so guessing from it opens the wrong pipeline and then writes to it. `PipelineStorageService.resolve(ref)` is now the only way in. An id that cannot be found is an error rather than an invitation to fall back; a name resolves against the authoritative listing, and a name shared by two pipelines is refused instead of guessed at. A miss no longer reaches for browser storage. `useLoadSpec` returns the resolved file, so `useSpecLifecycle` initialises synchronously and a resolve failure surfaces as a suspense error instead of a null file. Autosave with no open file now throws rather than reporting success and advancing its saved marker. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`getDefaultEditorPath(name)` could only produce a path, so every caller
handed the editor a display name and nothing else. With opaque storage keys
a name identifies nothing: two pipelines may share one, and the editor has
to guess.
`getDefaultEditorTarget(ref)` returns `{to, params, search}`, putting the
file id in the search params where it survives navigation; the `…Href`
variant covers the two `window.open` sites and the places that need a plain
string. Callers pass a ref, so a call site that learns the id later only has
to start supplying it.
Two places dropped the id even when it was in the URL: switching editor
versions rebuilt the path from scratch, and the v2-to-v1 redirect discarded
search entirely. Both now carry it through.
Undo history was keyed on the pipeline name in a Dexie primary key, where
two pipelines sharing a name replay each other's edits. It is keyed on the
file id now; the old table is dropped rather than migrated, because the
events only mean anything against the spec ids they were recorded from.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every read of the pipeline list went straight to the browser-local store, so a host-provided store was invisible to the dashboard, the name dialog, the metadata panels, the session stats and the v1 loader alike. Route them through the storage service instead. `pipelineOperations` gives the non-React callers the same service instance the provider hands to components, and a driver may now `describe` a key so a file resolved by name still carries its timestamps. The dashboard list filters and searches inside a spec that a remote listing does not carry, so a host-backed store gets the folder table. Session stats leave out the node-count distribution against a remote store: counting nodes would mean fetching every pipeline once a day.
camielvs
marked this pull request as draft
September 8, 2026 22:38
New Pipeline, save, save-as, import, clone-a-run and the v1 editor's save all wrote straight to the browser-local list, so with a host-provided store they created pipelines nobody could see and saved over ones nobody had opened. Route them through the service instead. `savePipelineByName` updates the pipeline that already has the name wherever it lives, rather than laying a second copy in the root store, and creates one when the name is new. Import now reports the identity of what it wrote, so the three places that navigate afterwards open that pipeline rather than whatever shares its name. Deleting is a fix, not a reroute: it removed the legacy row and the registry row but never asked the driver, so a pipeline deleted from the folder table stayed alive in its own store. It also reported success whatever happened; it now says when it failed. A path segment is classified as a run id only on a run path. The old 20-hex guess would mistake an opaque pipeline key for one, and the editor then loaded nothing at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rename them Renaming assumed the store keys a pipeline by its name, so renaming was moving the key. A store that hands out its own keys has no key to move — it takes the displayed name from the next spec written — and the driver could only throw. Make renaming optional on the driver instead. A file whose store cannot rename by key leaves the key alone, and the name arrives with the write that follows. Which is why the two are one operation for callers that only know names: split apart, the store would keep the old name and the caller would then create a second pipeline under the new one. The legacy editor's rename was the last writer still going straight to the browser-local list, and it now goes through the service like the rest. That leaves the list's rename with one caller and no use for its find-the-file-from-the-URL fallback. A name dialog no longer refuses a duplicate the store itself accepts, and no longer lists pipelines it has nothing to check them against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pipeline table read the browser's own store directly, so a host-provided store had nothing to show it and the route redirected to the folder table instead — swapping the page out from under anyone who clicked Pipelines. It reads through the storage layer now, like the folder table does. The listing says what pipelines exist but not what is in them, and the table searches inside them, so contents are fetched behind the rows a few at a time and kept against the version the store reports for each pipeline. A revisit re-reads only what actually changed; the first visit is the only one that pays. The flat list is the whole store rather than the root folder's share of it, because that is what this page has always shown and folders are the other page's idea. Two listings at once used to race to claim the same registry rows and the unique index failed the loser, taking down the listing over a row that already said what it wanted to say. Claiming is a transaction now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…vided store A user whose pipelines were all in this browser arrived at a host-backed store to find an empty library, with no way to get any of it across. Copy it, once, the first time the app runs against a host. Each pipeline is written under the name it already had: host writes upsert on the key they are given, so a run that dies halfway can simply be run again and a pipeline copied twice is overwritten rather than duplicated. Two tabs opening together must not both copy the library, so the claim is a transaction. A claim is only honoured for as long as its holder keeps making progress — otherwise one crashed tab would leave the migration permanently unrunnable — and a tab that finds a live claim waits rather than duplicating the work. It runs where the list would be, not in front of the app: a store that cannot be reached must not be able to lock anyone out of the editor. What fails is named, retriable, and skippable, because otherwise one pipeline the store refuses would block the page for good. Nothing is deleted. The browser keeps its copy, which is also what a build with no host still has to find. The pipeline table was also dropping a pipeline's id when opening it, falling back to resolving the name — which is not unique in a store that names nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A rejected write set an error and let the edit go. The editor was then the only copy of that work, with a 16px icon as the entire warning, and nothing would try again unless the user happened to keep typing — so anyone who stopped after a failure was quietly left unsaved. Hold the rejected edit instead and retry it: on a backoff, and immediately on coming back online or back to the tab, since either is a cheap sign the store might take it now. The indicator says the changes are still here rather than only that something went wrong. Two saves overlapping used to race to the store and the winner was whichever it finished last. A save that starts mid-write now waits, and the write already running picks up the newer content when it lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Storage mode was inferred from whether a host had loaded, which fails open: anything that stopped the host from installing sent the user to browser storage without a word, and their work went somewhere the deployment does not read from. VITE_PIPELINE_STORAGE says it outright. "host" requires one and says so plainly when the page did not provide it, rather than quietly writing elsewhere. "local" ignores one that loads anyway. Left unset it detects as before, which is what an unconfigured build wants and all the open-source build can do — it has no host to find. A store that hands out its own ids can now say so in the url, which carries the id alone: names are not unique there, so the name bought nothing a query param was not already paying for, and a path that is identity cannot go stale when a pipeline is renamed. Where names are the identity the url is unchanged. A single path segment is offered as both, so links written the other way round still open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he url the identity VITE_PIPELINE_STORAGE_BETA is a plain on/off: on means pipelines live outside the browser, and anything else — false, unset, a typo — means they live in it. Naming it a beta says what it is to anyone reading the source, and leaves the deployment holding the switch rather than inferring one from whether a host happened to load. Browser-stored pipelines go back to a url of just their name. Names are the identity there and each is held once, so the id said nothing the name did not and only made the url longer. The suites now pin the flag rather than inheriting it, because a developer's own .env would otherwise decide what they test. Host storage is settled while the app boots, so those e2e need a server of their own with the flag on, and they seed the browser's store directly — there is no local mode on that page to create a pipeline in, which is the situation the copy exists for. That seeding turned up a migration that could throw and leave the page saying "starting…" for good. It now says what went wrong and offers the same retry as a partial copy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every other flag the app reads is listed here; this one was only discoverable by finding the code that reads it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The registry and the spec cache are keyed on a storage key, which is only unique within one store, and browser storage keys a pipeline on its name — a name a host-provided store may well be holding too. Both stores also used the same root folder id, so nothing distinguished their rows at all. Opening a link to a host pipeline while the app was on browser storage therefore found the host's row, paired its identity with the browser's driver, and — because the key sent to the host is the pipeline's name — read and then saved over whichever browser pipeline happened to share it. The reverse held too, and a listing in one store deleted the other's rows as no longer present. Rows now say which store they describe and every lookup is scoped to the store this page load is using, so the two can hold the same name without meeting. Rows written before this cannot say: only a host-provided store reports a contentVersion, and a row filed in a folder can only be the browser's, which is enough to attribute them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A refused save only showed as a tooltip on a toolbar icon, so someone editing when the store went away saw nothing change and had every reason to think their work was being saved. It now says so across the top of the editor, for as long as the risk lasts, with the reason the store gave and a way to try again. The pipeline list said "you don't have any pipelines yet" when it could not read them at all, which invites building the same pipeline twice. It now says what happened and offers a retry, and only retries queries where a second attempt could answer differently. The parting write on close no longer goes straight to the file: sent on its own it raced whatever write was already running, and a slow store finishing them out of order left the older text stored. It goes through the same queue, which already holds only the newest edit — so a retry after a failure can never put back what a later save replaced. Tests now pin all three orderings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The copy only started on the pipeline list, so someone who landed anywhere else worked against a store that looked empty until they happened to open that tab. It now starts with the app. Nothing is shown from there — progress belongs where the pipelines are, and the list still reports anything that failed and offers to retry it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A store that keys pipelines itself has no folders, and the page still showed New Folder, Connect Folder and a row for a parent that cannot exist. Each was an action with nowhere to go. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An expired session was retried on the same ladder as a store that had merely gone away — every five, fifteen, thirty seconds, forever, while the work sat unsaved. It is now recognised as something only the person can resolve: the retrying stops and the editor says so. Signing in again means reloading, which discards whatever the store never took, so the download is offered before the reload rather than after it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A pipeline the store cannot produce fell to the generic "a UI element failed to render" icon, which names neither the pipeline nor the reason. A link written against a store the app is no longer using now reads as the miss it is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The architecture notes still described browser storage as the only store, a registry with one global key namespace, and functions that no longer exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ipelines Reacting to a failed listing was not enough. A listing that is slow, still working through its retries, or answered from cache leaves the library on screen saying nothing — so turning the deployment off and opening the pipelines page showed a full list of pipelines that could no longer be read or written. Both pipeline pages now ask whether the backend is answering, and say so in the same info box the runs list uses. The provider pings once at startup and never again, which cannot catch a deployment that goes down while a tab is open, so this re-asks on mount, on return to the tab, and on a slow interval while the page is there. Only in the beta: with the flag off nothing is asked and nothing changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The indicator only turned red once a write had already been refused, and read "auto-save enabled" until then — the reassuring state, at exactly the moment nothing could be saved. It now takes the same answer the pipeline list does: if the backend is not answering, the icon is red and the button does not invite a save that cannot happen. It goes back on its own when the next ping lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… backend The pipeline list and the auto-save indicator were reading the health of the configured execution backend, which is a different system: it can be switched off in Settings while pipelines carry on saving, and it can answer while the store that holds them has gone. Both readings were wrong in one direction or the other. Every call to the store already passes through one funnel, so that is where it is now learned — no health check against anything else, and nothing to go stale between calls. Only a failure to answer counts: "no such pipeline" is an answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pipelines were read and written through a store the embedding page installed on a window global, so this app knew nothing of the API behind it. That put the feature outside the very system built for pointing this editor at a backend: switching backends in Settings did not move the pipelines, switching one off did not stop them saving, and nobody running their own backend could serve them. They now go to `/api/users/me/pipelines` on whichever backend the app is configured against — the same place runs, secrets and components come from. A backend that serves those routes can hold pipelines for this editor, which is the point: the routes are a contract anyone can implement, not a private arrangement with one deployment. The driver interface was the seam, so the rest stands: the flat root, rows scoped to the store that wrote them, the copy out of browser storage, ids in the url, and every error surface. Where the backend is is read per request rather than frozen at startup, because that address is a setting and a setting can change while the app is open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s typing Two fixes to one gap. Rows written before pipelines moved onto the configured backend still say they belong to a store by its old name. Scoped lookups cannot see them, but their ids still occupy the primary key, so the next listing tried to add the same pipeline again and the collision took the pipeline page down with a raw IndexedDB error. They are renamed, and where a listing had already got part way through and written the row again, the newer one wins. Learning availability only from calls the app makes left an editor nobody is touching showing a healthy store through a restart. Now the backend is also pinged on mount, on return to the tab, and on an interval — which says something about the pipelines again, because they are served by the same backend the ping goes to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The retry was there, but only the backoff ladder and the browser's own online and focus events could trigger it — so an edit refused while the backend was down sat unsent for up to a minute after it came back, with the tab in view the whole time and nothing to click. The ping that turns the indicator red already knows the moment it is answering again. The editor now listens to it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An edit the backend would not take lived only in the editor that made it, and the most likely next thing someone does is leave the page to go and turn the backend back on. Closing the editor stopped the retrying; a full page load took the held edit with it. Either way the work was gone by the time the backend came back. Refused edits are now written down and sent when it next answers, whatever happened to the editor in between — including a reload, a different tab, or a session that ended hours earlier. The in-memory retry stays as the fast path. An editor that closes still holding work keeps trying too, and stands down if the same pipeline is reopened and saved elsewhere, so its older text cannot land on top of something newer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…path Pipelines predating the registry were indexed from Dexie's `ready` handler, which gates `open()`. The handler reads the legacy list out of another database first, and once it has yielded to a non-Dexie promise it can never touch this one again: every later call queues behind the open it is itself holding up. Awaiting the write there left the database opening forever, so the pipeline list never loaded for anyone who already had pipelines. Indexing now runs on its own, awaited by the folder listing that needs the rows, and claims only keys with no row yet inside a transaction — a listing running at the same time can no longer collide on the unique index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Trying the first-use copy more than once means emptying the backend first, which no user ever needs to do and there is otherwise no way to do from the app. The button leaves for the dashboard afterwards, because the copy starts from the pipeline list and would begin again the moment it finished. Browser storage is left alone: it is what the copy reads from, and its registry rows are the only record of which folder a pipeline is in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The notes still described a store an embedding page installed through a window global, which no longer exists. They now cover the routes the driver calls, how availability is decided, the outbox that holds a refused edit, and the two database traps this cost hours to find: a `ready` handler that deadlocks its own open, and renaming a value that lookups are scoped by. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
camielvs
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Beta, off by default. Today every pipeline you create lives in your own browser (IndexedDB), which means it is tied to one browser profile on one machine. This branch adds an optional second home: pipelines can live on the backend the app is configured against, so they follow you between machines.
A single build-time flag decides which:
With the flag off — the open-source build, local dev, and CI — the app behaves exactly as it does today. That is the regression risk this branch cares most about, and the existing e2e suite is the test for it.
Why the configured backend, rather than a seam
An earlier version of this branch routed storage through an object an embedding page installed on
window, specifically so the app would not know what it was talking to. That was the wrong instinct. The configurable-backend system exists so that people can run their own backend and point the app at it; storing pipelines is just another thing a backend can serve. So the app calls the routes directly, honours the configured backend URL, and anyone who implements these four routes can hold pipelines for this editor:GET /api/users/me/pipelines/all?page_size&page_tokenfile_pathfilters by prefixGET /api/users/me/pipelines?file_path=KEYPUT /api/users/me/pipelines?file_path=KEYDELETE /api/users/me/pipelines?file_path=KEYNo product, deployment, or internal system is named anywhere in the code, comments, tests, or type names.
src/services/pipelineStorage/ARCHITECTURE.mdis updated in this branch and is the best entry point for review.What a reviewer should know
When the flag is on, the backend is the only store — not an extra folder. The root folder is the backend folder, so every existing
folderId === null ? rootFolder : findFolderById(id)path lands on it unchanged. It is flat: no folders, so folder creation, the parent row and move-to-folder are hidden rather than offered and then refused.Pipelines get a name that is separate from their filing key. Browser storage keys a pipeline on its name, so the two have always been the same string. A backend hands out its own ids, so
PipelineFilehas adisplayNamethat falls back to the storage key. Two places quietly assumed one string and had to be fixed: the folder table (displayed and navigated with the same value) and the editor's reload-on-external-change check (which would have gone stale silently, with no error).The local registry is partitioned by store. The two stores share a namespace by nature — the key sent to the backend is the pipeline's local name — so an unscoped row was found by whichever store asked first. A backend link opened in the browser-storage build paired the backend's identity with the browser's driver, and then saved over whatever pipeline happened to share the name. Rows now carry which store they describe and every lookup is scoped to it. This also fixed two unreported bugs: a backend listing was deleting the browser's rows as "no longer present", and creating a browser pipeline was refused as a duplicate when the backend held that name.
Saved pipelines are cleaned up on the way out. This is the one change that affects everyone, flag or no flag. A pipeline document currently carries whatever the in-memory model accumulated, including things true about you rather than about the pipeline — favourited, owned, published-by — plus stale input values. Saves now pass through an allowlist, copying fields in their original order, so a pipeline with nothing to strip saves byte-for-byte identically.
Pipelines already in the browser are copied up once. Not per-save: a single claimed copy, keyed on each pipeline's local name, started wherever the app opens. The claim is a Dexie transaction honoured only while its holder keeps making progress, so two tabs cannot both run it and a tab that died does not block the next one. Nothing local is deleted — a build with the flag off still has to find its pipelines.
A store that has gone away is never represented by the last answer it gave. This took several rounds to get right. Availability is read both from the calls the app already makes and from a ping (a page nobody is touching makes no calls, so an editor left open through a restart would look healthy until the next keystroke), and the unavailable state renders ahead of a listing already in hand. In the editor, auto-save goes red and a banner says the work is held.
An edit the backend refuses outlives the editor. It goes to a durable outbox and is sent on the next recovery — the most likely next thing someone does is leave the page to go and fix the connection, and that must not be what loses the work. A reopened editor makes an older held edit stand down, so it can never land on top of newer text.
Commits
Readable in order, each self-contained. The arc: introduce the seam, move reads then writes onto it, make the store's own identity the editor url, then —
3bd127edd— replace the injected seam with the configured backend, followed by the availability, outbox and durability fixes found by driving it by hand.Related Issue and Pull requests
None. Nothing outside this repo is required any more: the routes are called directly.
Type of Change
Checklist
Verified locally:
pnpm validate(format, lint, typecheck, knip) and the unit suite — 239 files, 2561 tests. About 2,900 lines of new tests, includingPipelineStorageService,PipelineFolder, the registry's per-store scoping and its Dexie migrations, the spec cache, the copy's two-tab claim race, auto-save ordering and recovery, and the portable projection. Two Playwright projects: the default one with the flag off, and one with it on against a stub backend.Reviewer note on e2e: 17 canvas-dependent specs currently fail on my machine (the React Flow canvas never mounts). I verified the identical 17 fail on
masterwith this branch stashed, so it is environmental rather than a regression — but it does mean the flag-off suite is not currently giving full regression cover locally, and CI is the better signal.Test Instructions
With the flag off (the regression check that matters most). Just run the app. Create, rename, save, move and delete pipelines; open one in the editor and let auto-save run; check the folder views. Nothing should look or behave differently, and no extra folder should appear.
The one difference to look for: save a pipeline that came from the component library and inspect the YAML. Per-person fields (
favorited,owned,published_by,superseded_by) and stale inputvalues should be gone; everything else, including node positions, should be untouched and in the same order.With the flag on. Set
VITE_PIPELINE_STORAGE_BETA=truein.envand restart vite — it is a build-time value. Point the app at a backend serving the four routes above (Settings → Backend, orVITE_BACKEND_API_URL). Then:/pipelines. Pipelines already in your browser are copied up once; the list reports progress and offers a retry if any fail./pipelinesthat empties the backend and returns you to the dashboard, so step 1 can be tried again from scratch.To go back to browser-only: empty the backend first (turning the flag off only hides those pipelines), then set the flag to
falseand restart vite.