Skip to content

feat(llm): resolve tool support from the LiteLLM catalog - #8770

Merged
Lorenze Jay (lorenzejay) merged 15 commits into
viditostwal/litellm-catalog-lookupfrom
viditostwal/litellm-catalog-tool-capability
Nov 12, 2026
Merged

Lorenze Jay (lorenzejay) merged 15 commits into
viditostwal/litellm-catalog-lookupfrom
viditostwal/litellm-catalog-tool-capability

Conversation

@Vidit-Ostwal

@Vidit-Ostwal Vidit Ostwal (Vidit-Ostwal) commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Makes the LiteLLM model catalog decide whether a model supports tool use, with a user-controllable supports_tools setting that always takes precedence.

Stacked on #8742 (viditostwal/litellm-catalog-lookup). Merge #8742 first; this PR then retargets to main. If #8742 is squash-merged, rebase this branch onto main to drop the lookup commits.

Changes

  • BaseLLM.supports_tools: bool | None = None: available on every provider and on the LiteLLM path.
    • True / False: the user's value is final.
    • Not passed: the provider's class default applies. Anthropic and Bedrock keep True, and Gemini keeps its model-version check (>= 1.5). Existing behavior is unchanged.
    • Explicit None: the catalog's supports_function_calling decides, then the provider's default is the last resort.
    • OpenAI, Azure, Snowflake and LLM (LiteLLM path) default to None, so they use the catalog and then their existing heuristics (not is_o1_model, is_openai_model, model-prefix check).
  • BaseLLM._resolve_supports_function_calling(default): one helper for that order. Each provider's supports_function_calling() is a one-liner passing its own default.
  • Gemini: the validator fills the version default only when supports_tools is not passed. Previously it overwrote any value, including an explicit one. The same check is shared through _supports_tools_by_version.
  • Request gates: Anthropic, Gemini and Bedrock now send tools only when supports_function_calling() is true.
  • llm.py: the LiteLLM path honors an explicit supports_tools, then the catalog, then LiteLLM's own check.

Tests

  • tests/llms/test_function_calling_capabilities.py: explicit supports_tools wins over the catalog in both directions (Anthropic, Bedrock, Gemini, OpenAI, Azure); provider defaults hold when the value is not passed; explicit None uses the catalog and then the provider default (including Gemini by model version); request gates for each provider; real catalog lookups through the native classes and through LLM(model="provider/model").
  • A stub_catalog fixture patches the lookup instead of setting private attributes.
  • tests/llms: 1010 passed, 17 failed. The same 17 fail on main without this branch (multimodal, reasoning-effort forwarding, and the Anthropic/Bedrock "model not supported" tests).

Notes

  • Commits fed63da12 and 613c6517a were made with --no-verify: the mypy hook reports 4 LiteLLM-type errors in llm.py (ChatCompletionDeltaCustomToolCall, an unused type: ignore, an unfollowed import). They also occur without these changes.
  • The history includes a few commits that reverse each other while the supports_tools semantics were settled. Squash-merge is recommended.

🤖 Generated with Claude Code


Note

Medium Risk
Changes tool-use detection and API request shaping across all LLM providers; mis-resolution could drop tools or send them to unsupported models.

Overview
Introduces a unified way to decide whether an LLM may use tools: optional supports_tools on BaseLLM, then the LiteLLM catalog’s supports_function_calling, then each provider’s existing heuristic.

supports_tools is now bool | None (default None). When set to True or False, that choice wins. When unset, native providers still apply their class defaults (e.g. Anthropic/Bedrock default True); explicit None defers to the catalog before the provider default. OpenAI, Azure, Snowflake, and the LiteLLM LLM path use _resolve_supports_function_calling(default=...) so catalog data can override heuristics like “not o1” or “Azure OpenAI only.”

Request building on Anthropic, Gemini, Bedrock, and Azure now gates tools on supports_function_calling() instead of reading supports_tools directly—so catalog-driven “no tools” actually omits tool payloads (including enabling tools on non–OpenAI Azure models when supports_tools=True).

Gemini no longer overwrites supports_tools in its validator; version ≥1.5 remains the fallback via _supports_tools_by_version. Docs (EN/AR/KO/pt-BR) and AGENTS.md document the override semantics and tell contributors to call supports_function_calling(), not gate on the field.

Adds test_function_calling_capabilities.py plus Azure/Gemini test updates; a small CLI test tweak suppresses flow events for a disabled conversational flow.

Reviewed by Cursor Bugbot for commit 3a19f5f. Bugbot is set up for automated code reviews on this repo. Configure here.

@strix-security

strix-security Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 55 commits after the last Strix review (3814d7e). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: This PR adds a LiteLLM model-catalog lookup module and makes tool-use support resolvable per model, with an explicit `supports_tools` setting taking precedence over the catalog and each provider's existing default; Anthropic, Bedrock, Gemini and the LiteLLM path now gate tool payloads on that resolved capability.

Review summary

Reviewed all 13 changed files. The PR moves LiteLLM-catalog loading, model-name normalization and entry lookup into a new crewai/llms/model_catalog.py module, adds a supports_tools override on BaseLLM, and routes each provider's supports_function_calling() (plus the Anthropic, Bedrock and Gemini request gates) through the new resolver: explicit setting, then catalog entry, then provider default. These are capability-metadata decisions only — the consumers decide whether tool definitions are sent as native function calls or handled by the pre-existing text protocol, and none of them are authorization, sanitization or approval gates. The new lookup code performs in-memory work over a bundled package JSON resource with fixed filenames, no eval/exec/deserialization of untrusted input, no shell, SQL, template or network sinks, and no user-controlled filesystem paths; its regexes are non-nested and its normalization loops terminate. Semgrep on the changed production modules reported no issues. No security vulnerabilities introduced or exposed by this diff.

Review 1 on this PR · 3814d7e · requested Oct 29 11:24 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@Vidit-Ostwal
Vidit Ostwal (Vidit-Ostwal) changed the base branch from main to viditostwal/litellm-catalog-lookup October 29, 2026 11:30
@github-actions github-actions Bot added size/L and removed size/XL labels Oct 29, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 86cb77e. Configure here.

@Vidit-Ostwal
Vidit Ostwal (Vidit-Ostwal) force-pushed the viditostwal/litellm-catalog-lookup branch from 1c4d4cf to 1f0574e Compare November 4, 2026 13:10
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rop warning

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…alog

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ver the catalog

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ecides

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e defers to catalog

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…solver

Let _resolve_supports_function_calling accept and return None so LLM can
reuse it before falling back to litellm; name the default explicitly in
the Anthropic and Bedrock providers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Drop the validator assignment that always overwrote supports_tools so
Gemini resolves explicit setting, catalog, then version heuristic like
the other providers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
_prepare_completion_params attached tools only for OpenAI model names,
while supports_function_calling() now honors supports_tools and the
LiteLLM catalog. Non-OpenAI Azure models reporting tool support silently
sent requests without tools.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Vidit-Ostwal
Vidit Ostwal (Vidit-Ostwal) force-pushed the viditostwal/litellm-catalog-tool-capability branch from b1fa307 to bef8ff8 Compare November 4, 2026 13:42
@github-actions github-actions Bot added size/XL and removed size/L labels Nov 4, 2026
@github-actions github-actions Bot added size/L and removed size/XL labels Nov 4, 2026
@lorenzejay
Lorenze Jay (lorenzejay) merged commit 58b38d0 into viditostwal/litellm-catalog-lookup Nov 12, 2026
55 checks passed
@lorenzejay
Lorenze Jay (lorenzejay) deleted the viditostwal/litellm-catalog-tool-capability branch November 12, 2026 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants