Skip to content

feat(agent): make the conversational loop aware of its output schema - #2003

Merged
maxduu merged 8 commits into
mainfrom
feat-structured-output-in-prompt
Nov 19, 2026
Merged

maxduu merged 8 commits into
mainfrom
feat-structured-output-in-prompt

Conversation

@maxduu

@maxduu maxduu commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

See https://github.com/UiPath/uipath-agents-python/pull/900 for full context and testing.

In summary:
A conversational agent's structured output fields are filled by a separate LLM call after the loop ends. The loop itself never sees the output schema, so a builder who writes "set status to resolved in the output" in the system prompt gives the agent an instruction it cannot act on. The agent then either hunts for a tool that does not exist, or narrates the field values to the user.

This is the uipath-python half: the prompt itself. Wiring lives in uipath-agents-python, and the extraction-call handling in uipath-langchain-python.

What changes

get_chat_system_prompt gains an optional output_schema parameter. When present, the schema's JSON is appended to the conversational system prompt under a STRUCTURED OUTPUT FOR THIS TURN section that tells the model three things: the fields are recorded after its turn by a separate step, there is no tool for them, and the reply the user reads should not mention them.

Two supporting pieces:

  • get_conversational_output_hint_section(output_schema) renders the section; it returns "" for None, so callers that pass nothing behave exactly as before.
  • strip_conversational_output_hint_section(prompt) removes it again. The extraction call replays the exchange including the system message, where "you have no tool for these fields" would directly contradict the instruction telling the model to call set_conversational_output. Today a forced tool choice masks that contradiction; it would stop masking it as soon as models that reject forcing are downgraded to auto.

The strip is schema-independent: its markers come from the section constants used to render it, so the two cannot drift. An earlier version re-rendered the section from the schema to find it, which silently did nothing whenever the caller had narrowed the schema differently.

_GENERATE_OUTPUT_INSTRUCTION is also reworded to scope extraction to the current turn, so a value established in an earlier turn is not reported again when a flow loops back into the same agent.

Testing

39 tests in test_conversational_prompts.py, covering the section's presence and absence, the schema JSON being injected verbatim, and strip ∘ assemble restoring the bare prompt byte-for-byte.

End-to-end results are in the uipath-agents-python PR: 175 live runs across 7 authoring styles, 4 scenarios and 5 models. Summary — 8 replies leaked raw output JSON to the user without this section and 0 with it; 2 runs crashed inventing a tool and 0 with it; field accuracy unchanged.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the output-schema prompt changes.

No open issues 🎉

What changed in this PR

This PR adds output-schema awareness to the UiPath SDK’s conversational prompts, helping agents distinguish user-facing replies from structured fields recorded after each turn. Runtime wiring is handled in companion repositories.

Changes:

  • Adds an optional output-schema hint and a schema-independent helper to remove it before extraction.
  • Clarifies that structured output extraction applies only to the current turn.
  • Exports the new helpers and adds tests for rendering, removal, and prompt preservation.
File Description
packages/​uipath/​tests/​agent/​react/​test_conversational_prompts.py Tests schema hints, stripping behavior, and current-turn instructions.
packages/​uipath/​src/​uipath/​agent/​react/​conversational_prompts.py Adds schema-aware prompts and stripping support; refines extraction instructions.
packages/​uipath/​src/​uipath/​agent/​react/​__init__.py Exports the hint-rendering and stripping helpers.

Review effort: Balanced

maxduu and others added 2 commits November 18, 2026 23:11
The hard-wrapped section text becomes single lines; content is unchanged, and
the strip derives its markers from these same constants so it cannot drift.
Also formats the two over-long lines in strip_conversational_output_hint_section
that ruff format was already flagging.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →

@radu-mocanu radu-mocanu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!
don t forget to bump the patch version before merging

@sonarqubecloud

Copy link
Copy Markdown

@maxduu
maxduu merged commit 561cd63 into main Nov 19, 2026
186 checks passed
@maxduu
maxduu deleted the feat-structured-output-in-prompt branch November 19, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-integrations test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants