Repository navigation
feat(llm): provider-aware LiteLLM catalog lookup - #8742
Vidit Ostwal (Vidit-Ostwal) wants to merge 7 commits into
Conversation
Strix Security ReviewWarning This pull request has 41 commits after the last Strix review ( ✅ No open security findings. PR summary: Moves LiteLLM catalog loading and model-name normalization into a new `crewai/llms/model_catalog.py`, adds a provider-scoped `find_model_catalog_entry` lookup, and exposes the resolved entry on every LLM through a cached `BaseLLM.catalog_entry` property. Review summaryThis PR relocates the bundled LiteLLM catalog loading and model-name normalization into a new Ruled out (3)
Review 1 on this PR · Reviewed by Strix |
…zily Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
1c4d4cf to
1f0574e
Compare
* feat(llm): add provider-aware catalog lookup * feat(llm): read tool support from catalog * fix(llm): respect supports_tools opt-out and log dropped tools Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * refactor(llm): merge catalog function-calling helpers and drop tool-drop warning Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * feat(llm): add explicit supports_tools setting that overrides the catalog Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * refactor(llm): let provider and user supports_tools take precedence over the catalog Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * refactor(llm): drop provider supports_tools defaults so the catalog decides Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * refactor(llm): restore provider supports_tools defaults; explicit None defers to catalog Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * refactor(llm): route litellm function-calling check through shared resolver Let _resolve_supports_function_calling accept and return None so LLM can reuse it before falling back to litellm; name the default explicitly in the Anthropic and Bedrock providers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * refactor(gemini): let unset supports_tools defer to catalog then version Drop the validator assignment that always overwrote supports_tools so Gemini resolves explicit setting, catalog, then version heuristic like the other providers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs: document supports_tools and tool-support resolution Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(azure): gate tool attachment on supports_function_calling _prepare_completion_params attached tools only for OpenAI model names, while supports_function_calling() now honors supports_tools and the LiteLLM catalog. Non-OpenAI Azure models reporting tool support silently sent requests without tools. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(cli): suppress flow events in conversational fixture * test(cli): scope conversational flow event suppression * fix(llm): remove rebase conflict markers --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 58b38d0. Configure here.
| def supports_function_calling(self) -> bool: | ||
| """Check if the model supports function calling.""" | ||
| return not self.is_o1_model | ||
| return self._resolve_supports_function_calling(not self.is_o1_model) |
There was a problem hiding this comment.
OpenAI path still sends disabled tools
Medium Severity
supports_function_calling() can now be false from supports_tools=False or the LiteLLM catalog, and Azure, Anthropic, Gemini, and Bedrock already omit tools from the request in that case. OpenAI native completions and Responses still attach tools whenever they are passed, and the LiteLLM fallback still puts tools on the completion params. OpenAI-compatible and Snowflake inherit that completions path, so an explicit disable does not keep tools off the wire for those providers.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 58b38d0. Configure here.


Summary
Adds a provider-aware lookup into the bundled LiteLLM model catalog and exposes the resolved entry on every LLM as
BaseLLM.catalog_entry.Changes
crewai/llms/model_catalog.pybare_model_name,catalog_provider_prefixes), moved out ofcontext_window.py.find_model_catalog_entry(model, provider=None)returns the LiteLLM entry for an unambiguous match, orNone.azure_openai,aws,google,claude) and never cross provider catalogs.:0/-v1stripped), then normalized name. A match shared by several entries is left unresolved instead of picking one.BaseLLM.catalog_entry: lazy, cached property resolved frommodelandprovideron first access. It does not rely onmodel_post_init, so subclasses that override it keep working.context_window.py: now imports the shared catalog and helpers frommodel_catalog.py; behavior is unchanged. The duplicated:suffix/-vNstripping is shared betweenbare_model_nameand the deployment-name helper.Tests
tests/llms/test_model_catalog.py: provider scoping, aliases, qualified keys, Bedrock deployment ids, missing properties, unknown provider or model, empty model, and case-insensitive provider. The ambiguity case runs against a synthetic catalog fixture, so it won't break when the bundled LiteLLM data is refreshed.tests/llms/test_base_llm.py:catalog_entryuses the LLM's provider, isNonefor unknown models, survives amodel_post_initoverride, and is cached.tests/llms/test_model_catalog.py,test_base_llm.pyandtest_context_window.pypass (72 tests); ruff and mypy are clean. The full suite was not run.🤖 Generated with Claude Code
Note
Medium Risk
Changes how every native LLM decides whether to send tools to providers, which can alter runtime behavior for edge models (e.g. Azure Mistral vs GPT) though explicit
supports_toolsand broad test coverage mitigate regressions.Overview
This PR introduces a shared LiteLLM model catalog (
model_catalog.py) with provider-scopedfind_model_catalog_entry, refactors context-window sizing to use it, and exposes a cachedBaseLLM.catalog_entryon every LLM.Tool use is now resolved consistently: optional
supports_toolsonBaseLLMoverrides the catalog’ssupports_function_calling, then the provider default. Callers must usesupports_function_calling()—notsupports_tools—when deciding whether to attach tools. Native providers (Anthropic, Azure, Bedrock, Gemini, OpenAI, Snowflake) and the LiteLLM fallback path follow that order; request builders gatetools/toolConfigon the resolved capability instead of hard-coded flags (e.g. Azure non–OpenAI models can send tools when explicitly enabled).Docs (
AGENTS.md, LLM concept pages) describesupports_tools. Tests cover catalog lookup, catalog entry caching, cross-provider function-calling resolution, and Azure tool payload behavior.Reviewed by Cursor Bugbot for commit 58b38d0. Bugbot is set up for automated code reviews on this repo. Configure here.