Repository navigation
feat(llm): resolve tool support from the LiteLLM catalog - #8770
Conversation
Strix Security ReviewWarning This pull request has 55 commits after the last Strix review ( ✅ No open security findings. PR summary: This PR adds a LiteLLM model-catalog lookup module and makes tool-use support resolvable per model, with an explicit `supports_tools` setting taking precedence over the catalog and each provider's existing default; Anthropic, Bedrock, Gemini and the LiteLLM path now gate tool payloads on that resolved capability. Review summaryReviewed all 13 changed files. The PR moves LiteLLM-catalog loading, model-name normalization and entry lookup into a new Review 1 on this PR · Reviewed by Strix |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 86cb77e. Configure here.
1c4d4cf to
1f0574e
Compare
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rop warning Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…alog Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ver the catalog Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ecides Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e defers to catalog Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…solver Let _resolve_supports_function_calling accept and return None so LLM can reuse it before falling back to litellm; name the default explicitly in the Anthropic and Bedrock providers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Drop the validator assignment that always overwrote supports_tools so Gemini resolves explicit setting, catalog, then version heuristic like the other providers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
_prepare_completion_params attached tools only for OpenAI model names, while supports_function_calling() now honors supports_tools and the LiteLLM catalog. Non-OpenAI Azure models reporting tool support silently sent requests without tools. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
b1fa307 to
bef8ff8
Compare
58b38d0
into
viditostwal/litellm-catalog-lookup

Summary
Makes the LiteLLM model catalog decide whether a model supports tool use, with a user-controllable
supports_toolssetting that always takes precedence.Changes
BaseLLM.supports_tools: bool | None = None: available on every provider and on the LiteLLM path.True/False: the user's value is final.True, and Gemini keeps its model-version check (>= 1.5). Existing behavior is unchanged.None: the catalog'ssupports_function_callingdecides, then the provider's default is the last resort.LLM(LiteLLM path) default toNone, so they use the catalog and then their existing heuristics (not is_o1_model,is_openai_model, model-prefix check).BaseLLM._resolve_supports_function_calling(default): one helper for that order. Each provider'ssupports_function_calling()is a one-liner passing its own default.supports_toolsis not passed. Previously it overwrote any value, including an explicit one. The same check is shared through_supports_tools_by_version.supports_function_calling()is true.llm.py: the LiteLLM path honors an explicitsupports_tools, then the catalog, then LiteLLM's own check.Tests
tests/llms/test_function_calling_capabilities.py: explicitsupports_toolswins over the catalog in both directions (Anthropic, Bedrock, Gemini, OpenAI, Azure); provider defaults hold when the value is not passed; explicitNoneuses the catalog and then the provider default (including Gemini by model version); request gates for each provider; real catalog lookups through the native classes and throughLLM(model="provider/model").stub_catalogfixture patches the lookup instead of setting private attributes.tests/llms: 1010 passed, 17 failed. The same 17 fail onmainwithout this branch (multimodal, reasoning-effort forwarding, and the Anthropic/Bedrock "model not supported" tests).Notes
fed63da12and613c6517awere made with--no-verify: the mypy hook reports 4 LiteLLM-type errors inllm.py(ChatCompletionDeltaCustomToolCall, an unusedtype: ignore, an unfollowed import). They also occur without these changes.supports_toolssemantics were settled. Squash-merge is recommended.🤖 Generated with Claude Code
Note
Medium Risk
Changes tool-use detection and API request shaping across all LLM providers; mis-resolution could drop tools or send them to unsupported models.
Overview
Introduces a unified way to decide whether an LLM may use tools: optional
supports_toolsonBaseLLM, then the LiteLLM catalog’ssupports_function_calling, then each provider’s existing heuristic.supports_toolsis nowbool | None(defaultNone). When set toTrueorFalse, that choice wins. When unset, native providers still apply their class defaults (e.g. Anthropic/Bedrock defaultTrue); explicitNonedefers to the catalog before the provider default. OpenAI, Azure, Snowflake, and the LiteLLMLLMpath use_resolve_supports_function_calling(default=...)so catalog data can override heuristics like “not o1” or “Azure OpenAI only.”Request building on Anthropic, Gemini, Bedrock, and Azure now gates
toolsonsupports_function_calling()instead of readingsupports_toolsdirectly—so catalog-driven “no tools” actually omits tool payloads (including enabling tools on non–OpenAI Azure models whensupports_tools=True).Gemini no longer overwrites
supports_toolsin its validator; version ≥1.5 remains the fallback via_supports_tools_by_version. Docs (EN/AR/KO/pt-BR) andAGENTS.mddocument the override semantics and tell contributors to callsupports_function_calling(), not gate on the field.Adds
test_function_calling_capabilities.pyplus Azure/Gemini test updates; a small CLI test tweak suppresses flow events for a disabled conversational flow.Reviewed by Cursor Bugbot for commit 3a19f5f. Bugbot is set up for automated code reviews on this repo. Configure here.