Repository navigation
perf: reuse serializer wrapper models instead of rebuilding them on every call - #1335
Open
jason-schrader-openrouter wants to merge 1 commit into
Open
jason-schrader-openrouter wants to merge 1 commit into
jason-schrader-openrouter wants to merge 1 commit into
Conversation
…very call marshal_json and unmarshal call pydantic.create_model on every call, so each chat request built about 19 model classes. An SDK init hook now caches one wrapper model per type. Fixes #1056.
Contributor
|
I'll fix CI failures and address comments from users with write access that start with 'Devin'.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1056.
utils/serializers.pycallspydantic.create_model("Unmarshaller" | "Marshaller", body=(typ, ...), __config__=...)every timemarshal_jsonorunmarshalruns. As a result, eachchat.send/chat.send_asynccall builds and compiles about 19 throwaway model classes. That file is Speakeasy template code and gets overwritten on regeneration, so this PR fixes it from_hooks/, which regeneration doesn't overwrite:_hooks/serializer_cache.py.install()replacesserializers.create_modelwith a version that builds one model per(name, typ, config)and caches it withlru_cache.pydantic.create_model. If the template changes, the SDK keeps working; it just stops caching.init_hooks()inregistration.pycallsinstall(), so the cache is active as soon as anOpenRouterclient is created.Measured on 1.3.173 with a local stub server (Python 3.12, pydantic 2.14.3):
Responses were identical before and after.
tests/test_serializer_cache.pychecks that:chat.sendbuilds no new models;The first two tests fail if the
install()call is removed. Delete this hook once the upstream template caches these models itself.