Skip to content

feat(agent): keep the output-schema section out of the extraction call, and give advanced its system prompt - #1225

Merged
maxduu merged 8 commits into
mainfrom
feat-convo-output-prompt
Nov 19, 2026
Merged

maxduu merged 8 commits into
mainfrom
feat-convo-output-prompt

Conversation

@maxduu

@maxduu maxduu commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

See https://github.com/UiPath/uipath-agents-python/pull/900 for full context and testing.

Paired with UiPath/uipath-python#2003, which teaches the conversational loop's system prompt about the agent's output schema. This is the half that handles the second LLM call — the one that actually fills the fields.

What changes

1. Strip the output-schema section before extraction.

The extraction call replays the exchange's messages, system message included. That core conversational loop's system prompt now carries a section saying "the outputs are produced after your turn by a separate step; you have no tool for them". To avoid confusion for the output-generation call needs, we strip that portion from the output-generation step.

create_conversational_output_extractor rewrites the system message to remove that section by calling a helper function from uipath-python.

2. Give the advanced extraction call the agent's system prompt.

Before this fix, the advanced agent's output extraction step would not have access to the agent's system prompt.

Implementation note: the standard engine keeps its system prompt as a real SystemMessage at state.messages[0], while the advanced engine keeps it in a state field injected per model call by _RuntimeSystemPromptMiddleware. The extraction node replays state.messages, so builder guidance reached extraction on standard and never on advanced.

generate_conversational_output now resolves the prompt (static_prompt, else the state key written by capture_exchange_start — mutually exclusive by construction) and prepends it so it has the form [SystemMessage, User/Agent message(s)]

Testing

  • test_conversational_output_node.py — the section is stripped, non-section system messages pass through untouched, only index 0 is considered, empty histories tolerated.
  • test_conversational_advanced_agent_graph.py — five tests for the prepend: static prompt, resolved callable prompt, empty prompt prepends nothing, graph state does not gain the message, and the hint section is stripped before extraction. The last runs the real extractor against a mocked model rather than a stand-in, so it exercises the strip instead of assuming it.
  • Mutation-tested: each of those fails with the change reverted, and the empty-prompt test fails if the guard is removed, so none pass vacuously.
  • Full tests/agent/ suite passes. Four unrelated failures in tests/agent/multimodal/test_utils.py are missing optional extras (langchain_aws, langchain_google_vertexai) in the local dev environment, present before this branch.

maxduu and others added 5 commits November 8, 2026 21:47
…tion

Rename without_output_section to without_output_hint_section to match the
prompt helpers it calls, and return a new list instead of copying the sequence
and assigning to index 0. The standard path hands over state.messages directly,
so mutating the copy's first element risked writing the stripped prompt back
into graph state.

Also trims the docstring, whose detail now lives in the tests that pin the
behaviour.
The advanced engine keeps its system prompt in a state field, injected per
model call by _RuntimeSystemPromptMiddleware, so it never reaches
state.messages. The conversational output extraction node replays those
messages, which meant builder guidance in the system prompt informed the loop
and never reached the step that fills the output fields. The standard engine,
which keeps the prompt at messages[0], has always had it.

Resolve the prompt (static_prompt, else the state key written by
capture_exchange_start -- the two are mutually exclusive by construction) and
prepend it as a SystemMessage. The prepend builds a new list rather than
mutating state.messages: transform_output emits messages[initial_message_count:]
as the user-visible response messages, so a system message written into state
would surface in the chat.

The shared extractor already strips the loop's output-schema hint section from
any SystemMessage it receives, so the section does not leak into the extraction
call and contradict its instruction.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 19, 2026 03:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The tests lack a standard-path regression verifying that section hints are removed from LLM input without mutating the original state messages, allowing state corruption to go undetected.

1 open issue
What changed in this PR

This PR aligns conversational output extraction across the standard and advanced agent engines, preserving builder guidance while removing instructions that conflict with structured output generation.

Changes:

  • Strip the output-schema hint from the leading system message without mutating the original history.
  • Prepend the advanced agent’s static or resolved system prompt during extraction.
  • Add tests for prompt handling, extraction inputs, and graph-state preservation.
File Description
tests/​agent/​react/​test_conversational_output_node.py Tests message passthrough, instruction ordering, and history edge cases.
tests/​agent/​advanced/​test_conversational_advanced_agent_graph.py Tests prompt injection, state preservation, and hint removal.
src/​uipath_langchain/​agent/​react/​conversational_output_node.py Removes the output hint from extraction’s leading system message.
src/​uipath_langchain/​agent/​advanced/​agent.py Supplies the advanced agent’s system prompt to extraction.

Review effort: Balanced

assert sent[0] is system
assert sent[1] is human
assert "STRUCTURED OUTPUT FOR THIS TURN" not in sent[0].content
assert "Set ready_for_handoff once billing is confirmed." in sent[0].content
@maxduu
maxduu force-pushed the feat-convo-output-prompt branch from 632f77f to 75464cb Compare November 19, 2026 19:26
The node hands state.messages to the extractor directly, so a strip that
edited the system message in place would write the stripped prompt back into
graph state and the loop would silently lose its output-fields section.

No existing test covered this: the only section-bearing system message sat at
index 2, so the rebuild branch never ran with real graph state. Reverting the
rebuild to an in-place assignment passed the whole suite; it now fails exactly
this test.

Reported by Copilot on #1225.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@maxduu
maxduu merged commit e61f337 into main Nov 19, 2026
51 checks passed
@maxduu
maxduu deleted the feat-convo-output-prompt branch November 19, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants