Skip to content

perf: reuse serializer wrapper models instead of rebuilding them on every call - #1335

Open
jason-schrader-openrouter wants to merge 1 commit into
mainfrom
devin/1794545319-cache-serializer-models
Open

jason-schrader-openrouter wants to merge 1 commit into
mainfrom
devin/1794545319-cache-serializer-models

Conversation

@jason-schrader-openrouter

@jason-schrader-openrouter jason-schrader-openrouter commented Nov 13, 2026 •

Copy link
Copy Markdown

Summary

Fixes #1056.

utils/serializers.py calls pydantic.create_model("Unmarshaller" | "Marshaller", body=(typ, ...), __config__=...) every time marshal_json or unmarshal runs. As a result, each chat.send / chat.send_async call builds and compiles about 19 throwaway model classes. That file is Speakeasy template code and gets overwritten on regeneration, so this PR fixes it from _hooks/, which regeneration doesn't overwrite:

  • New file _hooks/serializer_cache.py. install() replaces serializers.create_model with a version that builds one model per (name, typ, config) and caches it with lru_cache.
  • Calls in any other shape, or with arguments that can't be hashed, go straight to pydantic.create_model. If the template changes, the SDK keeps working; it just stops caching.
  • init_hooks() in registration.py calls install(), so the cache is active as soon as an OpenRouter client is created.

Measured on 1.3.173 with a local stub server (Python 3.12, pydantic 2.14.3):

Models built per call CPU per call
Before 19 15–17 ms
After 0 4–5 ms

Responses were identical before and after.

tests/test_serializer_cache.py checks that:

  • creating a client installs the hook;
  • a second chat.send builds no new models;
  • cached models serialize the same way as uncached ones.

The first two tests fail if the install() call is removed. Delete this hook once the upstream template caches these models itself.

…very call

marshal_json and unmarshal call pydantic.create_model on every call, so each
chat request built about 19 model classes. An SDK init hook now caches one
wrapper model per type. Fixes #1056.
@devin-ai-integration

devin-ai-integration Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serializers rebuild pydantic models on every call: ~19 per chat.send_async, ~25 ms of CPU per request

1 participant