Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/docs/content/docs/agents/skills.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Selected skills appear as cards that you can open or remove. Editing depends on

At runtime, the Agent's system prompt includes an `<available_skills>` section with the selected names and descriptions. Sim adds a `load_skill` tool with a `skill_name` input. When the model calls it, the full instructions return as a tool response. Selecting a skill makes it available; the model decides whether to load it.

This uses standard tool calling across compatible models. Deep-research and evaluation models do not expose the ordinary Agent Tools and Skills controls.
This uses standard tool calling across compatible models. Evaluation models do not expose the ordinary Agent Tools and Skills controls.

## Use a skill in Chat

Expand Down
12 changes: 6 additions & 6 deletions apps/docs/content/docs/workflows/blocks/agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -188,17 +188,17 @@ Live tool-call chips stream for **OpenAI, Anthropic, Azure Anthropic, Google, Ve

| Provider | Streamed thinking | Models |
|----------|-------------------|--------|
| OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `gpt-6.2-astra`, `gpt-6.2-sol`, `gpt-6-astra`, `gpt-6.1-sol`, `gpt-6.1-terra`, `gpt-6-sol`, `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5-pro`, `gpt-5.5`, `gpt-5.4-pro`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.3-codex`, `gpt-5.2-pro`, `gpt-5.2`, `gpt-5.1`, `gpt-5-pro`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `o4-mini`, `o3`, `o3-mini`, `o1` |
| OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `gpt-6.2-astra`, `gpt-6.2-sol`, `gpt-6-astra`, `gpt-6.1-sol`, `gpt-6.1-terra`, `gpt-6-sol`, `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5-pro`, `gpt-5.5`, `gpt-5.4-pro`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.3-codex`, `gpt-5.2-pro`, `gpt-5.2`, `gpt-5.1`, `gpt-5-pro`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `o3` |
| Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `claude-fable-5-2`, `claude-fable-5-1`, `claude-fable-5`, `claude-sonnet-5-5`, `claude-sonnet-5`, `claude-opus-5-5`, `claude-opus-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-sonnet-4-5`, `claude-haiku-5-5`, `claude-haiku-4-5` |
| Azure OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `azure/gpt-6.1-sol`, `azure/gpt-6-sol`, `azure/gpt-6-luna`, `azure/gpt-6-astra`, `azure/gpt-5.6-sol`, `azure/gpt-5.6-terra`, `azure/gpt-5.6-luna`, `azure/gpt-5.5`, `azure/gpt-5.4-pro`, `azure/gpt-5.4`, `azure/gpt-5.4-mini`, `azure/gpt-5.4-nano`, `azure/gpt-5.2`, `azure/gpt-5.1`, `azure/gpt-5.1-codex`, `azure/gpt-5`, `azure/gpt-5-mini`, `azure/gpt-5-nano`, `azure/o3`, `azure/o4-mini` |
| Azure Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `azure-anthropic/claude-fable-5-1`, `azure-anthropic/claude-opus-5-5`, `azure-anthropic/claude-sonnet-5-5`, `azure-anthropic/claude-haiku-5-5`, `azure-anthropic/claude-opus-5`, `azure-anthropic/claude-opus-4-8`, `azure-anthropic/claude-opus-4-7`, `azure-anthropic/claude-opus-4-6`, `azure-anthropic/claude-opus-4-5`, `azure-anthropic/claude-sonnet-5`, `azure-anthropic/claude-sonnet-4-6`, `azure-anthropic/claude-sonnet-4-5`, `azure-anthropic/claude-opus-4-1`, `azure-anthropic/claude-haiku-4-5` |
| Azure OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `azure/gpt-6.1-sol`, `azure/gpt-6.1-terra`, `azure/gpt-6-sol`, `azure/gpt-6-luna`, `azure/gpt-6-astra`, `azure/gpt-5.6-sol`, `azure/gpt-5.6-terra`, `azure/gpt-5.6-luna`, `azure/gpt-5.5`, `azure/gpt-5.4-pro`, `azure/gpt-5.4`, `azure/gpt-5.4-mini`, `azure/gpt-5.4-nano`, `azure/gpt-5.2`, `azure/gpt-5.1`, `azure/gpt-5.1-codex`, `azure/gpt-5`, `azure/gpt-5-mini`, `azure/gpt-5-nano` |
| Azure Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `azure-anthropic/claude-fable-5-2`, `azure-anthropic/claude-fable-5-1`, `azure-anthropic/claude-fable-5`, `azure-anthropic/claude-opus-5-5`, `azure-anthropic/claude-sonnet-5-5`, `azure-anthropic/claude-haiku-5-5`, `azure-anthropic/claude-opus-5`, `azure-anthropic/claude-opus-4-8`, `azure-anthropic/claude-opus-4-7`, `azure-anthropic/claude-opus-4-6`, `azure-anthropic/claude-opus-4-5`, `azure-anthropic/claude-sonnet-5`, `azure-anthropic/claude-sonnet-4-6`, `azure-anthropic/claude-sonnet-4-5`, `azure-anthropic/claude-haiku-4-5` |
| Google | Summaries only | `gemini-4-neon`, `gemini-4-hydrogen`, `gemini-4-carbon`, `gemini-4-argon`, `gemini-3.8-flash`, `gemini-3.7-flash`, `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
| Vertex AI | Summaries only | `vertex/gemini-3.8-flash`, `vertex/gemini-3.7-flash`, `vertex/gemini-3.6-flash`, `vertex/gemini-3.5-flash-lite`, `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview`, `vertex/gemini-2.5-pro`, `vertex/gemini-2.5-flash`, `vertex/gemini-2.5-flash-lite` |
| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-flash`, `deepseek-v4-flash`, `deepseek-reasoner` |
| Vertex AI | Summaries only | `vertex/gemini-4-neon`, `vertex/gemini-4-hydrogen`, `vertex/gemini-4-carbon`, `vertex/gemini-4-argon`, `vertex/gemini-3.8-flash`, `vertex/gemini-3.7-flash`, `vertex/gemini-3.5-flash-lite`, `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview` |
| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-flash`, `deepseek-v4-flash` |
| xAI | Full thinking deltas | `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-0309` |
| xAI | Summaries only | `grok-4.7` |
| Cerebras | Full thinking deltas | `cerebras/qwen-3.8-27b` |
| Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.8-27b`, `groq/qwen/qwen3.6-27b` |
| Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.8-27b` |
| Meta | Not streamed | `muse-spark-1.3`, `muse-spark-1.1` |
| Kimi | Full thinking deltas | `kimi-k3`, `kimi-k2.6` |
| Kie | Summaries only | `kie/claude-sonnet-5-5`, `kie/claude-opus-5-5`, `kie/claude-opus-5`, `kie/claude-fable-5`, `kie/claude-sonnet-5`, `kie/claude-opus-4-8`, `kie/claude-opus-4-7`, `kie/claude-opus-4-6`, `kie/claude-sonnet-4-6`, `kie/claude-opus-4-5`, `kie/claude-sonnet-4-5`, `kie/claude-haiku-4-5`, `kie/gpt-6-1-sol`, `kie/gpt-6-astra`, `kie/gpt-6-sol`, `kie/gpt-6-luna`, `kie/gpt-5-6-sol`, `kie/gpt-5-6-terra`, `kie/gpt-5-6-luna`, `kie/gpt-5-5`, `kie/grok-4-6`, `kie/grok-4-5`, `kie/grok-4-3`, `kie/kimi-k3` |
Expand Down
1 change: 0 additions & 1 deletion apps/sim/app/(landing)/models/comparisons.ts
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,6 @@ const CAPABILITY_CHECKS: ReadonlyArray<{
supports: (model) => Boolean(model.capabilities.nativeStructuredOutputs),
},
{ label: 'computer use', supports: (model) => Boolean(model.capabilities.computerUse) },
{ label: 'deep research', supports: (model) => Boolean(model.capabilities.deepResearch) },
]

function buildCostSentence({ a, b }: ModelComparison): string {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ export function ModelDirectory() {
<h3 className='text-[18px] text-[var(--text-primary)]'>No matches found</h3>
<p className='mt-2 text-[var(--text-muted)] text-sm leading-[150%]'>
Try a provider name like OpenAI or Anthropic, or search for capabilities like
&nbsp;structured outputs, reasoning, or deep research.
&nbsp;structured outputs, reasoning, or tool choice.
</p>
</div>
) : (
Expand Down
37 changes: 16 additions & 21 deletions apps/sim/app/(landing)/models/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,15 @@ export function formatCapabilityBoolean(
return value ? positive : negative
}

function supportsCatalogStructuredOutputs(capabilities: ModelCapabilities): boolean {
return !capabilities.deepResearch
/**
* A retired model accepts no requests, and evaluation models answer native questions and reject a
* response format.
*/
function supportsCatalogStructuredOutputs(
capabilities: ModelCapabilities,
retired: boolean
): boolean {
return !retired && !capabilities.evaluation
}

export function getEffectiveMaxOutputTokens(capabilities: ModelCapabilities): number | null {
Expand Down Expand Up @@ -385,7 +392,7 @@ function formatModelDisplayName(providerId: string, modelId: string): string {
return displayName.replace(/^GPT (\d[\w.]*)/i, 'GPT-$1').replace(/\bGpt\b/g, 'GPT')
}

function buildCapabilityTags(capabilities: ModelCapabilities): string[] {
function buildCapabilityTags(capabilities: ModelCapabilities, retired: boolean): string[] {
const tags: string[] = []

if (capabilities.temperature) {
Expand All @@ -396,18 +403,14 @@ function buildCapabilityTags(capabilities: ModelCapabilities): string[] {
tags.push('Tool choice')
}

if (supportsCatalogStructuredOutputs(capabilities)) {
if (supportsCatalogStructuredOutputs(capabilities, retired)) {
tags.push('Structured outputs')
}

if (capabilities.computerUse) {
tags.push('Computer use')
}

if (capabilities.deepResearch) {
tags.push('Deep research')
}

if (capabilities.reasoningEffort) {
tags.push(`Reasoning ${capabilities.reasoningEffort.values.join(', ')}`)
}
Expand Down Expand Up @@ -438,10 +441,6 @@ function buildBestForLine(model: {
}): string | null {
const { pricing, capabilities, contextWindow } = model

if (capabilities.deepResearch) {
return 'Best for multi-step research workflows and agent-led web investigation.'
}

if (capabilities.reasoningEffort || capabilities.thinking) {
return 'Best for reasoning-heavy tasks that need more deliberate model control.'
}
Expand Down Expand Up @@ -484,7 +483,6 @@ function computeModelRelevanceScore(model: CatalogModel): number {
return (
(model.capabilities.reasoningEffort ? 10 : 0) +
(model.capabilities.thinking ? 10 : 0) +
(model.capabilities.deepResearch ? 8 : 0) +
(model.capabilities.nativeStructuredOutputs ? 4 : 0) +
(model.contextWindow ?? 0) / 100000
)
Expand All @@ -503,12 +501,13 @@ function compareModelsByRelevance(a: CatalogModel, b: CatalogModel): number {
const rawProviders = Object.values(PROVIDER_DEFINITIONS).map((provider) => {
const providerSlug = slugify(provider.id)
const providerDisplayName = provider.name
const providerCapabilityTags = buildCapabilityTags(provider.capabilities ?? {})
const providerCapabilityTags = buildCapabilityTags(provider.capabilities ?? {}, false)

const models: CatalogModel[] = provider.models.map((model) => {
const shortId = stripProviderPrefix(provider.id, model.id)
const mergedCapabilities = { ...provider.capabilities, ...model.capabilities }
const capabilityTags = buildCapabilityTags(mergedCapabilities)
const retired = model.sunset?.status === 'deprecated'
const capabilityTags = buildCapabilityTags(mergedCapabilities, retired)
const bestFor = buildBestForLine({
pricing: model.pricing,
capabilities: mergedCapabilities,
Expand All @@ -530,7 +529,7 @@ const rawProviders = Object.values(PROVIDER_DEFINITIONS).map((provider) => {
contextWindow: model.contextWindow ?? null,
releaseDate: model.releaseDate ?? null,
deprecated: !!model.sunset,
retired: model.sunset?.status === 'deprecated',
retired,
featured: model.featured ?? false,
recommended: model.recommended ?? false,
pricing: model.pricing,
Expand Down Expand Up @@ -825,7 +824,7 @@ export function buildModelFaqs(provider: CatalogProvider, model: CatalogModel):

export function buildModelCapabilityFacts(model: CatalogModel): CapabilityFact[] {
const { capabilities } = model
const supportsStructuredOutputs = supportsCatalogStructuredOutputs(capabilities)
const supportsStructuredOutputs = supportsCatalogStructuredOutputs(capabilities, model.retired)

return [
{
Expand Down Expand Up @@ -868,10 +867,6 @@ export function buildModelCapabilityFacts(model: CatalogModel): CapabilityFact[]
label: 'Computer use',
value: formatCapabilityBoolean(capabilities.computerUse),
},
{
label: 'Deep research',
value: formatCapabilityBoolean(capabilities.deepResearch),
},
{
label: 'Memory support',
value: capabilities.memory === false ? 'Disabled' : 'Supported',
Expand Down
68 changes: 13 additions & 55 deletions apps/sim/blocks/blocks/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import {
getBaseModelProviders,
getEvaluationModels,
getMaxTemperature,
getModelsWithDeepResearch,
getModelsWithoutMemory,
getModelsWithPromptCaching,
getModelsWithReasoningEffort,
Expand All @@ -38,10 +37,8 @@ const MODELS_WITH_REASONING_EFFORT = getModelsWithReasoningEffort()
const MODELS_WITH_VERBOSITY = getModelsWithVerbosity()
const MODELS_WITH_THINKING = getModelsWithThinking()
const MODELS_WITH_PROMPT_CACHING = getModelsWithPromptCaching()
const MODELS_WITH_DEEP_RESEARCH = getModelsWithDeepResearch()
const MODELS_WITHOUT_MEMORY = getModelsWithoutMemory()
const EVALUATION_MODELS = getEvaluationModels()
const MODELS_WITHOUT_CHAT_CONTROLS = [...MODELS_WITH_DEEP_RESEARCH, ...EVALUATION_MODELS]

// Helper function to get the tool ID from a block type
const getToolIdFromBlock = (blockType: string): string | undefined => {
Expand Down Expand Up @@ -272,7 +269,7 @@ Return ONLY the JSON array.`,
defaultValue: [],
condition: {
field: 'model',
value: MODELS_WITHOUT_CHAT_CONTROLS,
value: EVALUATION_MODELS,
not: true,
},
},
Expand All @@ -283,7 +280,7 @@ Return ONLY the JSON array.`,
defaultValue: [],
condition: {
field: 'model',
value: MODELS_WITHOUT_CHAT_CONTROLS,
value: EVALUATION_MODELS,
not: true,
},
},
Expand Down Expand Up @@ -355,16 +352,9 @@ Return ONLY the JSON array.`,
mode: 'advanced',
condition: () => ({
field: 'model',
value: (() => {
const deepResearch = new Set(MODELS_WITH_DEEP_RESEARCH.map((m) => m.toLowerCase()))
const allModels = Object.keys(getBaseModelProviders())
return allModels.filter(
(model) =>
supportsTemperature(model) &&
getMaxTemperature(model) === 1 &&
!deepResearch.has(model.toLowerCase())
)
})(),
value: Object.keys(getBaseModelProviders()).filter(
(model) => supportsTemperature(model) && getMaxTemperature(model) === 1
),
}),
},
{
Expand All @@ -377,16 +367,9 @@ Return ONLY the JSON array.`,
mode: 'advanced',
condition: () => ({
field: 'model',
value: (() => {
const deepResearch = new Set(MODELS_WITH_DEEP_RESEARCH.map((m) => m.toLowerCase()))
const allModels = Object.keys(getBaseModelProviders())
return allModels.filter(
(model) =>
supportsTemperature(model) &&
getMaxTemperature(model) === 1.5 &&
!deepResearch.has(model.toLowerCase())
)
})(),
value: Object.keys(getBaseModelProviders()).filter(
(model) => supportsTemperature(model) && getMaxTemperature(model) === 1.5
),
}),
},
{
Expand All @@ -399,16 +382,9 @@ Return ONLY the JSON array.`,
mode: 'advanced',
condition: () => ({
field: 'model',
value: (() => {
const deepResearch = new Set(MODELS_WITH_DEEP_RESEARCH.map((m) => m.toLowerCase()))
const allModels = Object.keys(getBaseModelProviders())
return allModels.filter(
(model) =>
supportsTemperature(model) &&
getMaxTemperature(model) === 2 &&
!deepResearch.has(model.toLowerCase())
)
})(),
value: Object.keys(getBaseModelProviders()).filter(
(model) => supportsTemperature(model) && getMaxTemperature(model) === 2
),
}),
},
{
Expand All @@ -419,7 +395,7 @@ Return ONLY the JSON array.`,
mode: 'advanced',
condition: {
field: 'model',
value: MODELS_WITHOUT_CHAT_CONTROLS,
value: EVALUATION_MODELS,
not: true,
},
},
Expand All @@ -431,21 +407,11 @@ Return ONLY the JSON array.`,
language: 'json',
condition: {
field: 'model',
value: MODELS_WITHOUT_CHAT_CONTROLS,
value: EVALUATION_MODELS,
not: true,
},
wandConfig: RESPONSE_FORMAT_WAND_CONFIG,
},
{
id: 'previousInteractionId',
title: 'Previous Interaction ID',
type: 'short-input',
placeholder: 'e.g., {{agent_1.interactionId}}',
condition: {
field: 'model',
value: MODELS_WITH_DEEP_RESEARCH,
},
},
{
...getModelFallbackSubBlock(),
condition: { field: 'model', value: EVALUATION_MODELS, not: true },
Expand Down Expand Up @@ -640,13 +606,5 @@ Return ONLY the JSON array.`,
description: 'Provider timing information',
},
cost: { type: 'json', description: 'Cost of the API call' },
interactionId: {
type: 'string',
description: 'Interaction ID for multi-turn deep research follow-ups',
condition: {
field: 'model',
value: MODELS_WITH_DEEP_RESEARCH,
},
},
},
}
20 changes: 0 additions & 20 deletions apps/sim/executor/handlers/agent/handler.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,6 @@ const mockResolveBedrockCredentialSession =
llmBedrockCredentialMockFns.mockResolveBedrockCredentialSession
mockImportWorkspaceFileSecretProvenanceForModelView.mockResolvedValue(true)
providersUtilsMockFns.mockGetProviderFromModel.mockReturnValue('mock-provider')
providersUtilsMockFns.mockIsDeepResearchModel.mockImplementation((model: string) =>
model.includes('deep-research')
)
providersUtilsMockFns.mockGetApiKey.mockReturnValue('mock-api-key')

process.env.NEXT_PUBLIC_APP_URL = 'http://localhost:3000'
Expand Down Expand Up @@ -1043,23 +1040,6 @@ describe('AgentBlockHandler', () => {
)
})

it('never falls back on a deep-research follow-up turn', async () => {
mockExecuteProviderRequest.mockRejectedValueOnce(new Error('overloaded'))

await expect(
handler.execute(mockContext, mockBlock, {
...baseInputs,
previousInteractionId: 'interaction-1',
fallbackModels: [{ model: 'claude-sonnet-5' }],
})
).rejects.toThrow('overloaded')
expect(mockExecuteProviderRequest).toHaveBeenCalledTimes(1)
expect(mockAgentLogger.info).toHaveBeenCalledWith(
'Fallback models skipped for a deep-research follow-up turn',
expect.objectContaining({ blockId: mockBlock.id })
)
})

it('gives a fallback its own key, the block key on the same provider, and nothing otherwise', async () => {
mockExecuteProviderRequest
.mockRejectedValueOnce(new Error('one'))
Expand Down
Loading
Loading