You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(llamaindex): let the LLM Gateway record model spans - #383
Trace-context headers: every UiPath model (UiPathOpenAI, UiPathChatBedrock / UiPathChatBedrockConverse, UiPathVertex) now sends x-uipath-traceparent-id and x-uipath-tracebaggage on each Gateway request. The Gateway then records the model call ("Model run", with its governance and guardrail spans) inside the agent's trace. Before this, LlamaIndex models sent no trace context.
No duplicate client span: OpenInference's client-side LLM span would duplicate "Model run", so it is no longer created. It is never started, not just dropped at export, so live tracking can't leave it behind as "running".
Opt-out:UIPATH_USE_CLIENT_LLM_SPANS=true (also 1 / yes) keeps the client-side LLM span and stops sending the traceparent; the baggage is still sent. This is the same variable as uipath-langchain, for agents whose models don't go through the Gateway. It is read on every call.
How
_trace_context.py: the env var and trace_context_headers(), called from each model's existing per-request hook (OpenAI's httpx transport, Bedrock's _modify_request, Vertex's _rewrite_request_for_gateway). Headers are built fresh for each request from the active span. If building them fails, the request goes out without them.
runtime/_llm_spans.py: LlamaIndex nests spans by its own span ids, not OTel context, so an LLM call without a span needs three things:
whatever it calls nests under its nearest ancestor that has a span;
its exit and events are ignored, rather than logged as Open span is missing;
current_span() (the provider registered with UiPathSpanUtils) gives the traceparent that ancestor's span, so the Gateway's spans land under the workflow step or agent that made the call.
These are subclasses of OpenInference's span and event handlers. skip_client_llm_spans() switches the instances instrument() built to them. A skipped call is kept only while it runs: it is forgotten when it exits or is dropped, which llama-index always reports, streams included. A chat or completion stream still emits events with the call's id after it returns (in-progress, LLM…EndEvent, ExceptionEvent). Those are ignored when OpenInference has no span for that id, so nothing is kept for a stream, and a stream that is never read can't leak.
runtime/factory.py: installs both.
Dependencies
uipath-platform>=0.3.2, <0.4.0 is the first release where the trace-context headers aren't behind the EnableTraceContextHeaders flag (uipath-python#1979). That requires uipath>=2.15.1, <2.16.0. The lock uses uipath 2.15.1 and uipath-platform 0.3.2 rather than today's 2.15.2 / 0.3.4, which CI's package proxy rejects as too new. The version is bumped to 0.6.3.
Tests
tests/runtime/test_trace_context.py (30 tests) runs a real workflow against a local tracer provider. It covers:
no client LLM span by default;
the traceparent names the calling step's span;
no missing-span warnings;
errors still propagate;
the env var values (true / TRUE / 1 / yes keep the span, false / 0 / empty skip it);
cleanup after normal, failing and streamed calls, including streams abandoned half-way or never read (async and sync);
late stream events of an unknown span are ignored, while other events of an unknown span are still reported as missing;
idempotence and factory wiring;
the headers on each of the three models.
Each mechanism was checked by breaking it on purpose, and the tests fail when it is broken.
testcases/template-agent/expected_traces.json no longer requires the client LLM span, which is gone by default. The same change was made for uipath-langchain.
Notes for reviewers
The span handling depends on OpenInference's private _SpanHandler / EventHandler. test_factory_installs_the_skip fails if an upgrade renames them, so client LLM spans can't silently come back.
Agents using a non-UiPath LlamaIndex model (e.g. native OpenAI) have no Gateway span to replace the client one. They lose their LLM span by default unless they set UIPATH_USE_CLIENT_LLM_SPANS=true. This matches uipath-langchain.
Embeddings are unchanged: they send no traceparent and keep their client spans.
The baggage source is source=llamaindex, following source=claude-sdk.
QA
Local robot QA will follow in a comment.
uipath-llamaindex/chat-models fails for UiPathChatBedrock only (Cannot instantiate typing.Union). This also happens on main and comes from anthropic 2.x (released Oct 21): llama-index-llms-bedrock pulls in llama-index-llms-anthropic 0.8.x, which calls anthropic.types.MessageParam(...), and MessageParam is now a Union. It is unrelated to this change.
UiPath LlamaIndex models (OpenAI, Bedrock, Vertex) now send the LLM Gateway
trace-context headers on every request, so the Gateway records the model call
("Model run", with its governance and guardrail spans) inside the agent's
trace. OpenInference's client-side LLM span would duplicate it, so the runtime
no longer creates one.
UIPATH_USE_CLIENT_LLM_SPANS=true keeps the client-side LLM span instead and
stops sending the traceparent (the baggage is still sent), for agents whose
models don't go through the Gateway. Same variable as uipath-langchain.
Requires uipath-platform>=0.3.2, where the trace-context headers are no longer
behind the EnableTraceContextHeaders flag, and so uipath>=2.15.2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The async-stream finalizer removes skipped-call registration before closure finishes, allowing cleanup events to reach OpenInference and produce missing-span warnings when a partially consumed stream is garbage collected.
This PR links LlamaIndex model requests to UiPath LLM Gateway traces, making Gateway-recorded model spans the default instead of duplicate client-side spans.
Changes:
Adds per-request trace headers to OpenAI, Bedrock, and Vertex models.
Suppresses client LLM spans while preserving ancestor context, with an environment-variable opt-out.
Adds tracing tests, updates expected traces, and bumps the package and required dependencies.
CI's package proxy rejects releases younger than its minimum age, and
uipath 2.15.2 and uipath-platform 0.3.4 were published today. 0.3.2 is the
first platform release without the EnableTraceContextHeaders flag, and
uipath 2.15.1 accepts it, so the floor moves to uipath>=2.15.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A half-read stream that was dropped lost its registration before it was
closed: CPython clears weak references before closing a generator, and
llama-index's stream wrapper emits an ExceptionEvent (GeneratorExit) while
closing, which then reached OpenInference as a missing span.
A skipped call is now forgotten when it returns or fails, or, for a chat or
completion stream, when its final event (end or exception) arrives, which the
wrapper emits exactly once when the stream is exhausted, fails or is closed.
Also builds the failing model outside pytest.raises in the tests (Sonar S5778).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stream that was returned but never read sends no further events, so its
registration, kept until the stream's final event, was never removed: one
entry leaked per abandoned stream.
A skipped call is now forgotten when it exits or is dropped, which
llama-index always reports, streams included. The events a stream emits
afterwards (in-progress, end, exception) are ignored when OpenInference has
no span for their id, so nothing is kept for a stream at all.
Tests cover both sides of that filter: late stream events of an unknown span
log nothing, and other events of an unknown span are still reported as
missing. They read OpenInference's logger directly rather than through
caplog, which other tests in the suite leave disabled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Custom HTTP clients can bypass trace-header injection while the client LLM span is still suppressed, leaving model calls unlinked from the agent trace.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Custom HTTP clients can break trace propagation while suppressing LLM spans
UiPathOpenAI lets http_client and async_http_client supplied through **kwargs replace the clients containing the trace-header transports. With a custom client that does not inject these headers, this check still suppresses the client LLM span, so the model call is no longer linked to the agent trace. Inject fresh trace headers for supplied clients too, or retain the client span when propagation is unavailable, and cover both overrides with regression tests.
Re Copilot's "Custom HTTP clients can break trace propagation while suppressing LLM spans" (_llm_spans.py:105):
That's right: UiPathOpenAI applies {**defaults, **kwargs}, so a caller's http_client / async_http_client replaces the UiPath transports and the traceparent isn't sent. But those transports also rewrite /openai/deployments/... to the Gateway's /completions path (llms/_openai.py:22-29), and that was already lost with a custom client before this PR. So custom clients aren't supported today, and the missing trace link is a side effect of that, not something this PR introduces.
Leaving it out of scope here. If custom clients should be supported, a follow-up can add both the headers and the URL rewrite to whatever client is passed (e.g. through httpx event_hooks), with tests for both overrides.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the LLM Gateway trace linking from uipath-langchain (uipath-langchain-python#1197) to
uipath-llamaindex.UiPathOpenAI,UiPathChatBedrock/UiPathChatBedrockConverse,UiPathVertex) now sendsx-uipath-traceparent-idandx-uipath-tracebaggageon each Gateway request. The Gateway then records the model call ("Model run", with its governance and guardrail spans) inside the agent's trace. Before this, LlamaIndex models sent no trace context.UIPATH_USE_CLIENT_LLM_SPANS=true(also1/yes) keeps the client-side LLM span and stops sending the traceparent; the baggage is still sent. This is the same variable as uipath-langchain, for agents whose models don't go through the Gateway. It is read on every call.How
_trace_context.py: the env var andtrace_context_headers(), called from each model's existing per-request hook (OpenAI's httpx transport, Bedrock's_modify_request, Vertex's_rewrite_request_for_gateway). Headers are built fresh for each request from the active span. If building them fails, the request goes out without them.runtime/_llm_spans.py: LlamaIndex nests spans by its own span ids, not OTel context, so an LLM call without a span needs three things:Open span is missing;current_span()(the provider registered withUiPathSpanUtils) gives the traceparent that ancestor's span, so the Gateway's spans land under the workflow step or agent that made the call.These are subclasses of OpenInference's span and event handlers.
skip_client_llm_spans()switches the instancesinstrument()built to them. A skipped call is kept only while it runs: it is forgotten when it exits or is dropped, which llama-index always reports, streams included. A chat or completion stream still emits events with the call's id after it returns (in-progress,LLM…EndEvent,ExceptionEvent). Those are ignored when OpenInference has no span for that id, so nothing is kept for a stream, and a stream that is never read can't leak.runtime/factory.py: installs both.Dependencies
uipath-platform>=0.3.2, <0.4.0is the first release where the trace-context headers aren't behind theEnableTraceContextHeadersflag (uipath-python#1979). That requiresuipath>=2.15.1, <2.16.0. The lock usesuipath2.15.1 anduipath-platform0.3.2 rather than today's 2.15.2 / 0.3.4, which CI's package proxy rejects as too new. The version is bumped to 0.6.3.Tests
tests/runtime/test_trace_context.py(30 tests) runs a real workflow against a local tracer provider. It covers:true/TRUE/1/yeskeep the span,false/0/ empty skip it);Each mechanism was checked by breaking it on purpose, and the tests fail when it is broken.
testcases/template-agent/expected_traces.jsonno longer requires the client LLM span, which is gone by default. The same change was made for uipath-langchain.Notes for reviewers
_SpanHandler/EventHandler.test_factory_installs_the_skipfails if an upgrade renames them, so client LLM spans can't silently come back.OpenAI) have no Gateway span to replace the client one. They lose their LLM span by default unless they setUIPATH_USE_CLIENT_LLM_SPANS=true. This matches uipath-langchain.source=llamaindex, followingsource=claude-sdk.QA
Local robot QA will follow in a comment.
uipath-llamaindex/chat-modelsfails forUiPathChatBedrockonly (Cannot instantiate typing.Union). This also happens onmainand comes fromanthropic2.x (released Oct 21):llama-index-llms-bedrockpulls inllama-index-llms-anthropic0.8.x, which callsanthropic.types.MessageParam(...), andMessageParamis now aUnion. It is unrelated to this change.🤖 Generated with Claude Code