Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions apps/docs/content/docs/mcp/tools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Tools
description: Discover an operation, inspect its request, and call it with the correct access.
---

Sim MCP exposes four tools for discovering and calling its supported JSON API operations. Search for an operation, inspect its inputs, then call the read or write tool named in its description.
Sim MCP exposes six tools for discovering and calling its supported JSON API operations. Search for an operation, inspect its inputs, then call the read or write tool named in its description, or run a program that makes many calls with a [code tool](#code-mode).

## Tools

Expand Down Expand Up @@ -72,13 +72,12 @@ correct its request.

`getWorkflowSource` returns a workflow as TypeScript source files with its `lock` and `base`. Edit the files and send them back, with the same `lock` and `base`, through `applyWorkflowSource`, first with `query: {"dryRun": true}`. One call replaces a batch of graph operations. Workflows as code is in preview; see [Workflows as code](/workflows/code).

## Code mode (beta)
## Code mode

For a task that takes many calls, such as paging through a list, filtering, or
updating every item that matches, the agent can write one program instead of
making each call itself. Only the program's return value comes back, so
intermediate results stay out of the model's context. Code mode is in beta and
appears only for accounts where it is enabled.
intermediate results stay out of the model's context.

| Tool | Does |
| --- | --- |
Expand Down Expand Up @@ -115,8 +114,10 @@ happened.

Each call runs through the API exactly as a call tool's would, with the same
credential, permissions, and rate limits. The program has no network access and
never sees your credential. A program runs for at most 60 seconds by default (up
never sees your credential. A program runs for at most 45 seconds by default (up
to 120 with `timeoutMs`), makes at most 200 calls, and returns at most 1 MiB.
One call's input may be at most 256 KB as JSON; send a larger body, such as a
big batch of rows, with `call_write_operation`.

## Related docs

Expand Down
70 changes: 41 additions & 29 deletions apps/sim/lib/api/mcp/__integration__/code-mode.integration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,13 @@ import { db } from '@sim/db'
import { apiKey, permissions, user, userTableDefinitions, workspace } from '@sim/db/schema'
import { deleteWorkspaceFixture, insertWorkspaceFixture } from '@sim/db/testing/workspace-fixtures'
import { authMock } from '@sim/testing/mocks/auth.mock'
import { featureFlagsMock, featureFlagsMockFns } from '@sim/testing/mocks/feature-flags.mock'
import { sleep } from '@sim/utils/helpers'
import { generateId } from '@sim/utils/id'
import { toRecord } from '@sim/utils/object'
import { eq } from 'drizzle-orm'
import { NextRequest } from 'next/server'
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'

vi.mock('@/lib/auth', () => authMock)
vi.mock('@/lib/core/config/feature-flags', () => featureFlagsMock)

import { hashApiKey } from '@/lib/api-key/crypto'
import { POST as simMcpPost } from '@/app/api/mcp/route'
Expand All @@ -28,6 +25,7 @@ const userId = generateId()
const workspaceId = generateId()
const writeKey = `sk-sim-fixture-${generateId()}`
const readKey = `sk-sim-fixture-${generateId()}`
const workspaceKey = `sk-sim-fixture-${generateId()}`

interface ToolResult {
isError?: boolean
Expand Down Expand Up @@ -130,44 +128,42 @@ describe('Sim MCP code mode in PostgreSQL and the isolate pool', () => {
type: 'personal',
access: 'read',
},
{
id: generateId(),
userId,
workspaceId,
name: 'Workspace',
key: workspaceKey,
keyHash: hashApiKey(workspaceKey),
type: 'workspace',
access: 'read',
},
])
})

beforeEach(() => {
featureFlagsMockFns.mockIsFeatureEnabled.mockImplementation(
async (flag: string, target?: unknown) =>
flag === 'mcp-code-mode' && toRecord(target).userId === userId
)
})

afterAll(async () => {
await deleteWorkspaceFixture(db, eq(workspace.id, workspaceId))
await db.delete(user).where(eq(user.id, userId))
await db.$client.end()
})

it('lists the code tools only when the flag is on for the caller', async () => {
const listed = await mcp(writeKey, { method: 'tools/list' })
it('runs programs for a workspace key, which has no person to key the sandbox by', async () => {
const listed = await mcp(workspaceKey, { method: 'tools/list' })
expect(listed.status, JSON.stringify(listed.body)).toBe(200)
const names = (listed.body.result as { tools: Array<{ name: string }> }).tools.map(
(t) => t.name
)
expect(names).toEqual(expect.arrayContaining(['execute_read_code', 'execute_write_code']))

featureFlagsMockFns.mockIsFeatureEnabled.mockResolvedValue(false)
const hidden = await mcp(writeKey, { method: 'tools/list' })
const hiddenNames = (hidden.body.result as { tools: Array<{ name: string }> }).tools.map(
(t) => t.name
const read = await runCode(
'execute_read_code',
`const page = await sim.api.listTables({ query: { workspaceId: ${JSON.stringify(workspaceId)} } })
return Array.isArray(page.data)`,
{ key: workspaceKey }
)
expect(hiddenNames).not.toContain('execute_read_code')
expect(hiddenNames).not.toContain('execute_write_code')
const refused = await mcp(writeKey, {
method: 'tools/call',
params: { name: 'execute_read_code', arguments: { code: 'return 1' } },
})
expect(refused.body.result).toMatchObject({
isError: true,
content: [{ text: expect.stringContaining('Tool execute_read_code not found') }],
})
expect(read.isError, JSON.stringify(read)).toBeFalsy()
expect(read.structuredContent?.result).toBe(true)
expect(read.structuredContent?.calls).toMatchObject([{ operation: 'listTables', status: 200 }])
})

it('writes through real routes and reads them back with paging, returning only the result', async () => {
Expand Down Expand Up @@ -195,9 +191,9 @@ describe('Sim MCP code mode in PostgreSQL and the isolate pool', () => {
'execute_read_code',
`
const found = []
let cursor
let cursor = null
do {
const page = await sim.api.listTables({ query: { workspaceId: ${JSON.stringify(workspaceId)}, limit: 1, ...(cursor ? { cursor } : {}) } })
const page = await sim.api.listTables({ query: { workspaceId: ${JSON.stringify(workspaceId)}, limit: 1, cursor } })
found.push(...page.data.map((t) => t.name))
cursor = page.nextCursor
} while (cursor)
Expand Down Expand Up @@ -272,6 +268,22 @@ describe('Sim MCP code mode in PostgreSQL and the isolate pool', () => {
})
})

it('refuses an input key outside params, query, body, and headers instead of dropping it', async () => {
const result = await runCode(
'execute_read_code',
`
try {
await sim.api.listTables({ query: { workspaceId: ${JSON.stringify(workspaceId)} }, limit: 1 })
return 'sent without its limit'
} catch (error) {
return { status: error.status, code: error.code }
}
`
)
expect(result.structuredContent?.result).toEqual({ status: 400, code: 'INVALID_CALL' })
expect(result.structuredContent?.calls).toEqual([])
})

it('passes a numeric path parameter, as signatures declare it, through to its route', async () => {
const result = await runCode(
'execute_read_code',
Expand Down
4 changes: 2 additions & 2 deletions apps/sim/lib/api/mcp/catalog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@ export const operationRequestInput = z
'Values for the bracketed segments of the operation’s path, by name, e.g. { "tableId": "..." }. IDs come from list operations such as listTables.'
),
query: z
.record(z.string(), z.union([z.string(), z.number(), z.boolean()]))
.record(z.string(), z.union([z.string(), z.number(), z.boolean(), z.null()]))
.optional()
.describe(
'Query-string values by name, as the query schema from describe_operation lists them, e.g. { "limit": 50 } or the cursor from the previous page.'
'Query-string values by name, as the query schema from describe_operation lists them, e.g. { "limit": 50 } or the cursor from the previous page. A null value is left out, so the last page’s null cursor ends paging.'
),
headers: z
.record(z.string(), z.string())
Expand Down
24 changes: 20 additions & 4 deletions apps/sim/lib/api/mcp/code-mode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,15 @@ import { SIM_API_CODE_MAX_TIMEOUT_MS } from '@/sandboxes/sim-api-code'

const logger = createLogger('SimMcpCodeMode')

const DEFAULT_TIMEOUT_MS = 60_000
/** Under the 60-second request timeout MCP clients default to, so a default run answers before its client gives up. */
const DEFAULT_TIMEOUT_MS = 45_000
Comment thread
waleedlatif1 marked this conversation as resolved.
Resolved
/** API calls one program may make; each is a full v2 request against the caller's rate limit. */
const MAX_CALLS_PER_RUN = 200
/** Calls one program may have in flight at once, so a `Promise.all` cannot fan out a whole page of requests. */
const MAX_CONCURRENT_CALLS = 8
const MAX_CODE_CHARS = 100_000
/** Slack past the isolate's own timeout, so a run that never queued still ends with the isolate's TimeoutError. */
const DEADLINE_GRACE_MS = 1000
const NOT_SENT = 'The program ended before this call was sent.'

export const codeModeInput = z
Expand All @@ -40,7 +43,7 @@ export const codeModeInput = z
.min(1)
.max(MAX_CODE_CHARS)
.describe(
'The body of an async function. Call operations with `await sim.api.<operation>(input)`, where input is { params?, query?, body?, headers? } exactly as describe_operation’s signature declares, and `return` a JSON-serializable value: only that value comes back. TypeScript is accepted; types are stripped, not checked. A failed call throws SimApiError with status, code, message, and details.'
'The body of an async function. Call operations with `await sim.api.<operation>(input)`, where input is { params?, query?, body?, headers? } exactly as describe_operation’s signature declares, and `return` a JSON-serializable value: only that value comes back. TypeScript is accepted; types are stripped, not checked. A failed call throws SimApiError with status, code, message, details, and requestId. One call’s input may be at most 256 KB as JSON; send a larger body with call_write_operation.'
),
timeoutMs: z
.number()
Expand Down Expand Up @@ -257,6 +260,9 @@ export async function runCodeMode(
}
}

const timeoutMs = input.timeoutMs ?? DEFAULT_TIMEOUT_MS
/** Bounds the whole run, time spent waiting for a sandbox worker included, so the caller hears back within its timeout. */
const deadline = AbortSignal.timeout(timeoutMs + DEADLINE_GRACE_MS)
let bytes: Buffer | undefined
let runError: unknown
try {
Expand All @@ -266,8 +272,8 @@ export async function runCodeMode(
{
ownerKey,
callSimApi,
signal: context.signal,
timeoutMs: input.timeoutMs ?? DEFAULT_TIMEOUT_MS,
signal: AbortSignal.any([context.signal, deadline]),
timeoutMs,
}
)
} catch (error) {
Expand All @@ -288,6 +294,16 @@ export async function runCodeMode(
summary('cancelled')
return failure({ name: 'AbortError', message: 'The program was cancelled.' }, calls)
}
if (deadline.aborted) {
summary('timeout')
return failure(
{
name: 'TimeoutError',
message: `The program did not finish within ${timeoutMs}ms, including time spent waiting for a sandbox.`,
},
calls
)
}
if (runError instanceof SandboxUserCodeError) {
summary(runError.name === 'TimeoutError' ? 'timeout' : 'error')
return failure({ name: runError.name, message: runError.message }, calls)
Expand Down
4 changes: 2 additions & 2 deletions apps/sim/lib/api/mcp/dispatch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ type V2RouteHandler = (
export interface McpOperationCall {
operation: V2McpOperationName
params?: Record<string, string | number>
query?: Record<string, string | number | boolean>
query?: Record<string, string | number | boolean | null>
body?: unknown
headers?: Record<string, string>
}
Expand Down Expand Up @@ -195,7 +195,7 @@ export async function dispatchMcpRequest(

const url = new URL(resolved.path, getBaseUrl())
for (const [name, value] of Object.entries(call.query ?? {})) {
url.searchParams.set(name, String(value))
if (value !== null) url.searchParams.set(name, String(value))
}

const hasBody = contract.method !== 'GET' && contract.body !== undefined
Expand Down
36 changes: 8 additions & 28 deletions apps/sim/lib/api/mcp/route-handler.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { type Principal, resolvePrincipalSubjectUserId } from '@sim/auth/principal'
import { type Principal, requirePrincipalSubjectUserId } from '@sim/auth/principal'
import { isPlainRecord } from '@sim/utils/object'
import type { NextRequest } from 'next/server'
import { v2McpOperations } from '@/lib/api/application/operations'
Expand Down Expand Up @@ -26,7 +26,6 @@ import {
} from '@/lib/api/server/routes/v2-json-route'
import type { OAuthAccessTokenOptions } from '@/lib/auth/oauth-access-token'
import { type ApplicationOperation, requireOAuthOperationScope } from '@/lib/core/application'
import { isFeatureEnabled } from '@/lib/core/config/feature-flags'
import { isSameOrigin } from '@/lib/core/utils/validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { v2CaughtOrchestrationError, v2Error } from '@/app/api/v2/lib/response'
Expand Down Expand Up @@ -69,7 +68,7 @@ async function toolCallOperation(
if (name !== TOOL_NAMES.read && name !== TOOL_NAMES.write) return null
if (!isPlainRecord(args) || typeof args.operation !== 'string') return null
const resolved = await resolveOperation(
args.operation,
args.operation.trim(),
name === TOOL_NAMES.read ? 'read' : 'write'
)
if ('error' in resolved) return null
Expand All @@ -78,27 +77,12 @@ async function toolCallOperation(
}

/**
* Whether a message's answer depends on the code-mode flag: the instructions
* and tool list mention the code tools, and a call may run one.
* Names the caller for the sandbox pool's per-caller fairness: the person
* calling, or the workspace a workspace key belongs to.
*/
function readsCodeMode(message: Record<string, unknown>): boolean {
if (message.method === 'initialize' || message.method === 'tools/list') return true
if (message.method !== 'tools/call' || !isPlainRecord(message.params)) return false
const { name } = message.params
return name === CODE_TOOL_NAMES.read || name === CODE_TOOL_NAMES.write
}

/**
* The code tools when the flag is on for this caller, keyed for the sandbox
* pool's per-caller fairness by the person or workspace calling.
*/
async function codeModeFor(principal: Principal): Promise<{ ownerKey: string } | null> {
const userId = resolvePrincipalSubjectUserId(principal)
const workspaceId = principal.kind === 'workspace_api_key' ? principal.workspaceId : undefined
if (!(await isFeatureEnabled('mcp-code-mode', { userId, workspaceId }))) return null
if (userId) return { ownerKey: `user:${userId}` }
if (workspaceId) return { ownerKey: `workspace:${workspaceId}` }
return null
function sandboxOwnerKey(principal: Principal): string {
if (principal.kind === 'workspace_api_key') return `workspace:${principal.workspaceId}`
return `user:${requirePrincipalSubjectUserId(principal)}`
}

/** Browsers may only reach the server from Sim's own origins (DNS-rebinding protection). */
Expand Down Expand Up @@ -130,11 +114,7 @@ export function createSimMcpHandlers() {
credential: readMcpCredentialHeaders(request.headers),
audience: simMcpAudience(),
},
{
codeMode: readsCodeMode(parsed.data.body)
? await codeModeFor(admission.auth.principal)
: null,
}
sandboxOwnerKey(admission.auth.principal)
)
return await serveStatelessMcp(server, request, parsed.data.body)
} catch (error) {
Expand Down
27 changes: 8 additions & 19 deletions apps/sim/lib/api/mcp/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,7 @@ const INSTRUCTIONS = `Sim is the AI workspace where teams build, deploy, and man
2. Read its input schemas with describe_operation.
3. Run it with the tool search_operations names: call_read_operation for operations that only read, call_write_operation for everything else.

Most operations take a workspaceId; listWorkspaces returns the workspaces you can use. Put path parameters in params, query-string values in query, and the JSON request body in body. Responses use the Sim API envelope ({ "data": ... }); list operations page with limit and cursor. Streaming options are not supported over MCP.`

const CODE_MODE_INSTRUCTIONS = `
Most operations take a workspaceId; listWorkspaces returns the workspaces you can use. Put path parameters in params, query-string values in query, and the JSON request body in body. Responses use the Sim API envelope ({ "data": ... }); list operations page with limit and cursor. Streaming options are not supported over MCP.

For a task that takes more than one or two calls (paging, filtering, joining, or changing many items), write one program instead: describe_operation returns each operation's TypeScript signature, and ${CODE_TOOL_NAMES.read} or ${CODE_TOOL_NAMES.write} runs an async function body that calls await sim.api.<operation>(input) and returns only what you need. Intermediate results stay out of your context. ${CODE_TOOL_NAMES.read} refuses every operation that needs write access.`

Expand Down Expand Up @@ -80,26 +78,20 @@ const describeInput = z.object({ operation: operationName }).strict()
/** Shared because a read is not always a GET: searching and querying post their filter as JSON. */
const callInput = operationRequestInput.extend({ operation: operationName }).strict()

interface SimMcpServerOptions {
/**
* Registers the code-mode tools for this caller; `ownerKey` names the caller
* for the sandbox pool's per-caller fairness. `null` leaves them out.
*/
codeMode: { ownerKey: string } | null
}

/**
* The Sim MCP server for one HTTP request. A request owns its server, so no
* credential outlives the request that presented it.
*
* Four tools cover the whole v2 API instead of one tool per operation: a
* catalog of 200-odd tools would overflow most clients' tool limits and spend
* A handful of tools cover the whole v2 API instead of one tool per operation:
* a catalog of 200-odd tools would overflow most clients' tool limits and spend
* the model's context on schemas it never uses. Reads and writes are separate
* tools so a client can approve reads once and still confirm every change.
* tools, for single calls and for programs alike, so a client can approve reads
* once and still confirm every change. `ownerKey` names the caller for the
* sandbox pool's per-caller fairness when it runs a program.
*/
export function createSimMcpServer(
context: Omit<McpDispatchContext, 'signal'>,
options: SimMcpServerOptions
ownerKey: string
): McpServer {
const baseUrl = getBaseUrl()
const server = new McpServer(
Expand All @@ -122,7 +114,7 @@ export function createSimMcpServer(
},
],
},
{ instructions: options.codeMode ? INSTRUCTIONS + CODE_MODE_INSTRUCTIONS : INSTRUCTIONS }
{ instructions: INSTRUCTIONS }
)

async function call(
Expand Down Expand Up @@ -224,9 +216,6 @@ export function createSimMcpServer(
async (input, extra) => call('write', input, extra.signal)
)

if (!options.codeMode) return server
const { ownerKey } = options.codeMode

async function runCode(
kind: 'read' | 'write',
input: z.infer<typeof codeModeInput>,
Expand Down
Loading
Loading