Skip to content

Serve runtime-observed output shapes for schemaless tools - #1759

Merged
RhysSullivan merged 2 commits into
mainfrom
shape-memory
Aug 27, 2026
Merged

Serve runtime-observed output shapes for schemaless tools#1759
RhysSullivan merged 2 commits into
mainfrom
shape-memory

Conversation

@RhysSullivan

Copy link
Copy Markdown
Collaborator

Muscle memory for code mode: most tools declare no output schema, so tools.describe.tool() renders data: unknown and models guess response shapes (or dump whole payloads). This learns shapes from live responses and serves them where nothing was declared.

Learnexecutor.execute folds every successful dynamic-tool payload into a bounded inferred shape (packages/core/sdk/src/shape-inference.ts): field names and broad types only, never values; wide objects collapse to additionalProperties so data-bearing keys (emails, ids) never persist; depth/width/union/size all capped. Merging across calls makes missing fields optional and unions divergent types.

Storeplugin_storage under a reserved system id (executor.shape-memory): already migrated in every host, tenant/owner-scoped, and untouched by catalog refresh (which wipes tool rows). In-memory read-through cache per executor; a row write happens only when a new observation changes the merged shape, so stable APIs converge to zero writes.

Servetools.schema substitutes the observed shape only when neither the catalog row nor projectToolSchema declares an output schema, rendered through the existing JSON-Schema→TS compiler. ToolSchemaView gains outputSchemaSource: "observed" + outputSchemaObservations, and describe.tool adds outputTypeScriptNote telling the model the fields are reliable but possibly incomplete (the compact TS render drops schema descriptions). The execute skill text documents the note.

Evidence — new e2e scenario shape-memory.test.ts passes on selfhost and cloud: schemaless OpenAPI tool describes as data: unknown, one live call teaches { issues: { id: number; title: string; open: boolean; }[]; total: number; }, next describe serves it with the provenance note. The cloud pass spans three separate HTTP requests, so it proves the cross-session persistence path, not just the in-memory cache. Unit: 14 inference tests, 2 executor integration tests, 1 invoker cold→warm test.

Out of scope (follow-ups) — MCP tools: the MCP plugin always synthesizes a CallToolResult envelope schema, so its generic structuredContent slot needs an envelope-aware splice in the MCP plugin's projection. Search results still carry no type signatures (separate change on the Tool row). No TTL on observed shapes: continued use keeps them fresh; an unused tool's shape can go stale after an upstream API change.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 26, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 9f8f477 Commit Preview URL

Branch Preview URL
Aug 27 2026, 12:27 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 26, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 9f8f477 Aug 27 2026, 12:28 AM

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@pkg-pr-new

pkg-pr-new Bot commented Aug 26, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@1759

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@1759

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@1759

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@1759

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@1759

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@1759

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@1759

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@1759

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@1759

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@1759

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@1759

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@1759

executor

npm i https://pkg.pr.new/executor@1759

commit: 9f8f477

@RhysSullivan
RhysSullivan marked this pull request as ready for review August 27, 2026 05:55
@RhysSullivan
RhysSullivan merged commit 30cfb4d into main Aug 27, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant