Repository navigation
feat(guardrails): group coded-agent guardrail spans like low-code agents [AL-712] - #1219
daniel-joica wants to merge 3 commits into
Conversation
Each guardrail middleware hook is its own LangGraph node, so a coded agent's
trace showed one node per guardrail per hook, with no grouping, no action spans
and nothing for deterministic rules.
- Stage containers ("LLM input guardrail check", ...) shared across the separate
guardrail nodes: the first check of a stage opens one, later guardrail nodes
of that stage reuse it, the next unrelated node closes it. The empty guardrail
node spans are hidden from the export.
- A guardrailAction span (Block, Log, ...) per action, linked to its evaluation.
- A client guardrailEvaluation span for deterministic rules.
The API is unchanged. Grouping only turns on when a guardrail middleware is in
use and a trace manager is available.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Overlapping guardrail evaluations can cause a shared tool-stage container to close prematurely, leaving slower checks outside their parent span and preventing late Block results from marking it as failed.
1 open issue
What changed in this PR
This PR aligns coded-agent guardrail traces with low-code agents by grouping checks into stage containers, without changing how middleware is configured.
Changes:
- Adds shared stage containers, lifecycle handling, and filtering of redundant guardrail node spans.
- Records guardrail actions and deterministic evaluations, with tracing tests.
- Integrates grouping into the runtime factory and bumps the version to 0.20.3.
| File | Description |
|---|---|
uv.lock |
Updates the package version to 0.20.3. |
tests/guardrails/middlewares/test_grouping.py |
Tests grouping, lifecycle, filtering, and concurrent runs. |
tests/guardrails/middlewares/test_action_span.py |
Tests action attributes, linkage, errors, and interrupts. |
src/uipath_langchain/runtime/factory.py |
Registers grouping and the span filter. |
src/uipath_langchain/guardrails/middlewares/deterministic.py |
Adds evaluation and action tracing to deterministic checks. |
src/uipath_langchain/guardrails/middlewares/_spans.py |
Implements guardrail spans and shared container lifecycle. |
src/uipath_langchain/guardrails/middlewares/_base.py |
Wraps checks and actions and registers guardrail nodes. |
pyproject.toml |
Bumps the project version to 0.20.3. |
Review effort: Balanced
| return None # nothing would ever close it | ||
|
|
||
| name, span_type = _STAGES[(scope, stage)] # type: ignore[index] | ||
| key = (parent.span_id, span_type) |
There was a problem hiding this comment.
In create_agent, LangGraph runs each tool call in its own tools span (one per Send), and tool containers are keyed by that span, so parallel calls never share one. Added test_overlapping_checks_on_parallel_tool_calls_stay_in_their_container (3 parallel calls, deliberately overlapping 50 ms checks): three containers, every check and action ends inside its own.
…-712] Addresses review on #1219: a regression test with three parallel tool calls and deliberately overlapping checks (each call has its own tools span, so its own container). Also Sonar: collect the closer's owners before deleting them, one throwing call per pytest.raises, and split a composite assertion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…L-712] On Windows before Python 3.13 the clock advances about every 15 ms, so the next sibling's start can be stamped a tick before the last check ended. Each stage now records when its last check finished, and the closer ends the container at the later of the two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|




Part of AL-712, together with UiPath/uipath-python#1997, which removes the duplicate client
evaluate_guardrailspan. This PR works with the publisheduipath-platform; it needs no platform change.What
In a coded agent, every guardrail middleware hook is its own LangGraph node. Seven guardrails gave about seven guardrail nodes around each model call: no grouping, no record of the action taken, and nothing at all for deterministic rules. Low-code agents group checks into stage containers and record each action.
This PR makes coded traces match low-code's, without changing how guardrails are written (
middleware=[*UiPathPIIDetectionMiddleware(...), ...]):Container names and span types are the low-code ones (
agentPre/Post,llmPre/Post,toolPre/PostGuardrails).How
All logic is in the new
guardrails/middlewares/_spans.py:guardrail_stage(scope, stage)wraps each check (evaluation and action). The first check of a stage creates a shared container next to the guardrail nodes (insidetoolsfor tool checks); later guardrail nodes of that stage reuse it. It's lazy: a stage where every check is skipped leaves nothing.guardrail_aware(provider)wraps the runtime's current span provider so a running check reports its container as the current span.@tracedspans, the action span and the Guardrails Service's span (through the trace headers) all land in it.GuardrailStageCloser(span processor) ends a container when a sibling that isn't this stage's guardrail node starts (at that sibling's start time, but never before its last check ended: coarse clocks such as Windows before Python 3.13 can stamp the sibling a tick early), or when its parent ends.is_exported(span filter in the factory's trace settings) hides the now-empty guardrail node spans: only OpenInference spans with a registered guardrail node name, only while grouping is on.guardrailAction:Block,Log, ...) withguardrailName,reason,severityLevel, andguardrailSpanIdlinking it to its evaluation.guardrailEvaluationspan (guardrailType=custom,validationResult,details), and their action links to it.execution.idfrom the node (their parent is non-recording), so execution-based lookups (evals) include them.Opt-in by use:
runtime/factory.pycallsenable_guardrail_grouping(trace_manager, get_current_span), which only registers the provider wrapper and offers the trace manager. A closer is attached, and grouping turns on, once a guardrail middleware is built (guardrails_in_use()). Agents without guardrails, and low-code agents (whose factory subclasses this one), get no closer and no filtering. When the trace manager shuts down, the closer unregisters itself and ends anything still open; the next run adds a new one.Other changes:
_base.py: checks run inguardrail_stage, actions inguardrail_action_span; message hooks register their node names.deterministic.py: a_checkmethod (evaluation span, then action) inside the tool stage. Behaviour is unchanged.Known limits
model, not inside anLLM callspan as in low-code:before_model,modelandafter_modelare separate nodes. Wrapping the model call (wrap_model_call) could add that level later.shutdown()clears the shared processor list (uipath-core), which also silences the other's closer. Today's CLI flows use one at a time.Tests
test_action_span.py(8): Block error span with all attributes, linked and parented; Log with severity; no span on a pass; an escalation pause is not an error; missing service id omitted; action names.test_grouping.py(21): nodes of one stage share a container that ends exactly atmodel's start, and never before its last check;execution.idcopied; a new stage gets its own container; a node in between splits it; LLM output; a block marks the container failed; nothing to check means no container; tool input and output containers with exact boundaries; off without a closer, under an invalid span, and without guardrail middlewares; one closer per trace manager, back after shutdown; tool-only guardrails turn it on; only OpenInference guardrail node spans are hidden; realcreate_agent+ OpenInference with 2 parallel tool calls and 2 concurrent runs (no orphans, nothing left open), and 3 parallel tool calls with deliberately overlapping checks (each call has its owntoolsspan, so its own container, and every check ends inside it).🤖 Generated with Claude Code