types/client: relax think type to support model-defined thinking levels - #744
Merged
Merged
Conversation
Models like Qwen 3.8 advertise custom thinking levels (such as 'xhigh') through /api/show thinking metadata, and the Ollama Go backend accepts any bool or string for ThinkValue. Previously, think was restricted to Optional[Union[bool, Literal['low', 'medium', 'high']]], causing Pydantic ValidationError in ChatRequest and GenerateRequest when passing supported levels like 'xhigh' or 'max'. Relaxing the annotation to Optional[Union[bool, str]] allows all valid model-defined thinking levels.
ParthSareen
self-requested a review
September 29, 2026 01:13
Member
|
Thank you for this! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #743
What & Why
Models like Qwen 3.8 advertise custom thinking levels (such as
'xhigh') through Ollama's/api/showthinking capabilities metadata (introduced in ollama/ollama#18473), and the Ollama Go backend natively accepts any boolean or string forThinkValue(types/model/thinking.go).Previously,
thinkwas constrained toOptional[Union[bool, Literal['low', 'medium', 'high']]]across request models and client methods. BecauseChatRequestandGenerateRequestare Pydantic models validated before requests are dispatched, passing valid model-defined levels like'xhigh'(or'max') resulted in a PydanticValidationError:This PR relaxes the
thinkannotation toOptional[Union[bool, str]] = Noneacross all 14 annotation sites in_types.pyand_client.py, matching the Go backend and allowing any valid model-defined string thinking level while preserving full backward compatibility.Changes
ollama/_types.py: UpdatedGenerateRequest.thinkandChatRequest.thinkfromLiteral['low', 'medium', 'high']tostr.ollama/_client.py: Updated all 12generateandchatoverloads and implementations inClientandAsyncClientto match.tests/test_client.py:test_generate_think_annotation_matches_chatto verify annotation parity acrossClient.chat,Client.generate,AsyncClient.chat, andAsyncClient.generate.test_client_chat_with_think_levelverifyingclient.chatserialization and response handling with string thinking levels (think='xhigh').tests/test_type_serialization.py:test_think_model_defined_levels_serializationverifying serialization of model-defined levels ('xhigh','max','low','medium','high').test_think_boolean_serializationverifying boolean thinking levels.Testing
All 104 tests pass:
Linter and formatter check clean: