Repository navigation
fix(documents): split Yjs state that exceeds the storage value limit - #869
Draft
posthog[bot] wants to merge 2 commits into
Draft
posthog[bot] wants to merge 2 commits into
posthog[bot] wants to merge 2 commits into
Conversation
SQLite-backed Durable Object storage rejects a value over 2 MB, and widget blocks can push a document's Yjs state past it. DocumentSession then throws SQLITE_TOOBIG on every persist, so AI reads fail and onSave skips the Postgres checkpoint. States up to 1 MB stay a single value at the update key. Larger states are split into 1 MB parts behind a manifest at that key, and each write removes the parts of the previous generation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 6b295967-a1c2-4c8a-9794-448858ca72f8
|
Important Review skippedBot user detected. To trigger a single review, invoke the
|
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
Only this module writes the manifest, so the version field and the full manifest guard add no protection. The part count follows from the total size. The shrink test now checks that the large write created parts first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 6b295967-a1c2-4c8a-9794-448858ca72f8
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
workspace_read_itemscall on that document fails withSQLITE_TOOBIG.DocumentSession.persistYDocwrites the full Yjs state as one Durable Object value. SQLite-backed storage rejects a value over 2 MB.Warning
onSaveuses the same write. When it throws, the Postgres checkpoint does not run, so edits to a large document can stop saving.Origin
Changes
document-yjs-persistence.ts:document-session:yjs-update. Small documents keep the same storage layout.onLoad,persistYDoc,applyEditsandundoDocumentEditReceiptuse the new read and write helpers.putof a Yjs state over 2 MB throwsSQLITE_TOOBIG. It also shows that the chunked write round-trips that state and removes the old parts when the state gets smaller.SQLITE_TOOBIGTesting
vp test --run --project=workers src/features/workspaces/documents/: passvp test --run --project=node src/features/workspaces/documents/: passvp check(format, lint, typecheck): passpnpm verify(ciBuildincluded).Agent context
beforeDocumentabove 1.5 MB, so AI-edit undo is not available on very large documents.Closes #868
Created with PostHog Desktop from this inbox report.
🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.