Fixes float32->float64 precision widening a.k.a. float bloat

[binarymax]

• I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

Fixes the precision-widening float32-->float64 bug for embeddings, aka "float bloat".

OpenAI (and other inference providers) return embeddings as float32. The SDK widens them to float64. The extra precision is erroneous, and incurs significant performance overhead on downstream use.

Additional context & links

https://bonsai.io/blog/float-bloat

Reproduction examples showing expected, before patch, and after patch:

EXPECTED: Use curl to verify float32 is returned from the model API directly

curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Bonsai",
    "model": "text-embedding-ada-002",
    "encoding_format": "float"
  }'

Expected response contains float32 precision for the embedding values:

{ "object": "list", "data": [ { "object": "embedding", "index": 0, "embedding": [ -0.0076202806, -0.0032867694, 0.0033270288, 0.0018809146, -0.019414647, ... ] } ], "model": "text-embedding-ada-002-v2", "usage": { "prompt_tokens": 3, "total_tokens": 3 } }

Use this simple Python script to use the SDK (before and after):

from openai import OpenAI

client = OpenAI(api_key=f"{OPENAI_API_KEY}")

print(
    client.embeddings.create(
        model="text-embedding-ada-002",
        input="Bonsai",
    )
)

BEFORE PATCH: returns precision-widened float64 instead of float32:

CreateEmbeddingResponse(data=[Embedding(embedding=[-0.007620280608534813, -0.0032867693807929754, 0.003327028825879097, 0.0018809145549312234, -0.019414646551012993, ...], index=0, object='embedding')], model='text-embedding-ada-002-v2', object='list', usage=Usage(prompt_tokens=3, total_tokens=3))

AFTER PATCH: returns float32 as expected:

➭ uv run --locked python scratch_test.py
CreateEmbeddingResponse(data=[Embedding(embedding=[-0.0076202806, -0.0032867694, 0.0033270288, 0.0018809146, -0.019414647, ...], index=0, object='embedding')], model='text-embedding-ada-002-v2', object='list', usage=Usage(prompt_tokens=3, total_tokens=3))

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff8ebc7add

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Avoid per-coordinate string round trips

For default-format embedding requests, this now formats and reparses every coordinate in Python, adding a temporary string allocation per value. In a local check with one 3,072-dimensional vector, this conversion took roughly 1.67 ms versus 0.05 ms for the previous tolist() conversion, and the cost scales across every vector in a batch; the mirrored async parser has the same issue. This can add seconds of SDK-side CPU work to large embedding batches, counteracting the performance goal of this change, so the shortening should be implemented without a Python string round trip for every coordinate.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bbe6593140

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Wire the precision-normalizing parser into requests

For default-format embedding requests, this newly defined callback is never invoked: the request still passes partial(_parse_embedding_response, ...) as its post_parser on line 144, so responses continue through the unchanged decoder and retain the widened representations this commit is intended to remove. The asynchronous implementation has the same disconnect; implement the normalization in the shared handwritten parsing helper and keep this generated resource delegating to it.

AGENTS.md reference: AGENTS.md:L3-L8

Useful? React with 👍 / 👎.