Skip to content

feat(guardrails): record deterministic rule checks as spans - #1221

Merged
daniel-joica merged 4 commits into
mainfrom
feat/guardrail-deterministic-spans
Nov 20, 2026
Merged

daniel-joica merged 4 commits into
mainfrom
feat/guardrail-deterministic-spans

Conversation

@daniel-joica

@daniel-joica daniel-joica commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

2 of 3 (#1220 is merged; this is now based on main):

  1. feat(guardrails): record each guardrail action as a span [AL-712] #1220 (merged): a span for each guardrail action.
  2. This PR: a span for each deterministic rule check.
  3. Grouping checks into stage containers (based on this one).

What

UiPathDeterministicGuardrailMiddleware rules run in the agent, so nothing recorded their check: a rule that fired only left a log line. Low-code agents record a client-side guardrailEvaluation span for their deterministic guardrails.

Each rule check now gets that span, and the action span on a violation (recorded since #1220) links to it:

tools
├─ London Weather Rule   (guardrailEvaluation, guardrailType=custom)  VALIDATION_FAILED
├─ Log                   (guardrailAction)  guardrailSpanId → London Weather Rule
└─ get_weather

Attributes: guardrailName, guardrailDescription, guardrailType=custom, validationResult (PASSED or VALIDATION_FAILED), details (the rule result's reason).

How

  • _spans.py: guardrail_evaluation_span(name, description) and set_validation_result(span, result), plus span_guid(span), which gives the span id as a GUID, the form guardrailSpanId uses. guardrail_action_span takes an optional evaluation_span_id and falls back to result.span_id.
  • deterministic.py: the rule check now runs inside the evaluation span, and its span id is passed to the action span. The empty-rules case is still handled in _evaluate_rules, and a block is still converted to AgentRuntimeError. Behaviour is unchanged.
  • Version 0.20.6 → 0.20.7.

Tests

  • New test_deterministic_spans.py (3): a failed rule records its evaluation with every attribute and the action links to it; a passing rule records its evaluation only; a blocking rule's action span is an error, its evaluation is not.
  • Full suite: 4350 passed, 3 skipped. ruff and ruff format are clean; mypy is clean on the guardrails package.

🤖 Generated with Claude Code

Development Package

  • Use uipath pack --nolock to get the latest dev build from this PR (requires version range).
  • Add this package as a dependency in your pyproject.toml:
[project]
dependencies = [
  # Exact version:
  "uipath-langchain==0.20.7.dev1012216343",

  # Any version from PR
  "uipath-langchain>=0.20.7.dev1012210000,<0.20.7.dev1012220000"
]

[[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
publish-url = "https://test.pypi.org/legacy/"
explicit = true

[tool.uv.sources]
uipath-langchain = { index = "testpypi" }

[tool.uv]
override-dependencies = [
    "uipath-langchain>=0.20.7.dev1012210000,<0.20.7.dev1012220000",
]

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The review identifies no active issues requiring changes to the deterministic guardrail tracing update.

No open issues 🎉

What changed in this PR

Adds client-side tracing for deterministic guardrail evaluations, continuing the guardrail observability work from #1220. Evaluation spans record validation outcomes and link to actions taken on violations.

Changes:

  • Adds evaluation-span helpers, result attributes, and GUID-formatted span IDs.
  • Integrates tracing into deterministic checks while preserving action handling and block-error conversion.
  • Adds tests for failed, passing, and blocking checks; bumps the package to 0.20.4.
File Description
uv.lock Updates the locked package version to 0.20.4.
tests/​guardrails/​middlewares/​test_deterministic_spans.py Tests evaluation attributes, action links, and span status.
src/​uipath_langchain/​guardrails/​middlewares/​deterministic.py Records evaluation spans and links violation actions.
src/​uipath_langchain/​guardrails/​middlewares/​_spans.py Adds evaluation-span, validation-result, and span-ID helpers.
pyproject.toml Bumps the project version to 0.20.4.

Review effort: Balanced

Base automatically changed from feat/guardrail-action-spans to main November 19, 2026 12:49
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@daniel-joica
daniel-joica force-pushed the feat/guardrail-deterministic-spans branch from 13cf3c6 to 6fa85ec Compare November 19, 2026 13:00
@daniel-joica daniel-joica changed the title feat(guardrails): record deterministic rule checks as spans [AL-712] feat(guardrails): record deterministic rule checks as spans Nov 19, 2026
…ectly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the guardrail tracing changes.

No open issues 🎉

Review effort: Balanced

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@daniel-joica

Copy link
Copy Markdown
Contributor Author

QA:

  • Before:
Screenshot 2026-11-19 at 16 26 23
  • After:
Screenshot 2026-11-19 at 16 23 22

…inistic-spans

# Conflicts:
#	pyproject.toml
#	uv.lock
@sonarqubecloud

Copy link
Copy Markdown

@daniel-joica
daniel-joica merged commit 3c337fd into main Nov 20, 2026
51 checks passed
@daniel-joica
daniel-joica deleted the feat/guardrail-deterministic-spans branch November 20, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants