Skip to content

Commit 849ee66

Browse files
committed
docs(workflows): scope large data limits to stored values and object reads
1 parent 577073c commit 849ee66

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

‎apps/docs/content/docs/workflows/large-data.mdx‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,16 @@ Sim stores block outputs larger than 8 MiB with the run and loads them only when
99

1010
| Limit | Size |
1111
| --- | --- |
12-
| One block output | 64 MiB |
13-
| One value loaded into Function code at once | 16 MiB |
12+
| One stored value | 64 MiB |
13+
| One stored value loaded by a block at once | 16 MiB |
1414
| One file read by Function code | 64 MiB |
1515
| One `sim.files` read or chunk | 10 MiB |
1616

17-
These are hard limits. A block that exceeds one fails with an `Execution memory limit exceeded` error that names the size and the limit.
17+
Sim stores large fields and array chunks as separate values, so a whole block output can exceed 64 MiB when no single piece does. A block that exceeds a limit fails with an `Execution memory limit exceeded` error that names the size and the limit.
1818

1919
## Reference fields, not whole outputs
2020

21-
Use `<api.data.items[0].id>` instead of `<api.data>`. Sim loads a large output only as far as the path requires, so a small field inside a large output stays cheap to read.
21+
Use `<api.data.items[0].id>` instead of `<api.data>`. Sim stores large arrays in chunks, so an indexed path loads only the chunk it reads. A field inside a large object still loads that whole object, so the object must fit the 16 MiB limit; when it does not, have the block that produces it return less.
2222

2323
Only JavaScript Function code without imports can receive a whole large value; Sim loads it when the code runs, up to 16 MiB. Agent, API, and other block inputs, Python and Shell code, and JavaScript with imports must reference a smaller field. A reference to an object that contains a large value fails with `contains nested large values`; reference the nested field instead.
2424

0 commit comments

Comments
 (0)