Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
650 changes: 383 additions & 267 deletions .upstreamer/eval-report.md

Large diffs are not rendered by default.

27 changes: 26 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ hooks.on(HookName.SessionEnd.value, HookEntry(handler=lambda payload, ctx: print
result = call_model(client, {"model": model, "input": prompt, "tools": tools, "hooks": hooks})
```

`SessionStart` fires once per run with a config summary; `SessionEnd` fires once with aggregated `total_usage` (summed across every `PostModelCall`) and is guaranteed to fire — and any pending async hook work drained — even when a no-tools stream raises. `PreToolUse` can block a call (`{"block": "reason"}`) or mutate its input (`{"mutated_input": {...}}`); `PermissionRequest` can pre-empt the approval gate with `{"decision": "allow" | "deny" | "ask_user"}`; `Stop` can force the loop to keep going past a `stop_when` hit with `{"force_resume": True, "append_prompt": "..."}`. A `HooksManager` instance is safe to share across concurrent `call_model` runs — session identity is threaded per emit, not stored as manager-level mutable state.
`SessionStart` fires once per run with a config summary; `SessionEnd` fires once with aggregated `total_usage` (summed across every `PostModelCall`) and is guaranteed to fire — and any pending async hook work drained — even when a no-tools stream raises. `PreToolUse` can block a call (`{"block": "reason"}`) or mutate its input (`{"mutated_input": {...}}`). It runs once per call, after the initial approval check and before execution, so a mutated input is validated against the tool's `input_schema` and re-gated by any input-dependent approval source (a callable `require_approval`, tool- or call-level); a call that pauses on that second gate is persisted with its effective arguments and `pre_tool_use_applied=True`, and the resume does not run `PreToolUse` again. `PermissionRequest` can pre-empt the approval gate with `{"decision": "allow" | "deny" | "ask_user"}`; `Stop` can force the loop to keep going past a `stop_when` hit with `{"force_resume": True, "append_prompt": "..."}`. A `HooksManager` instance is safe to share across concurrent `call_model` runs — session identity is threaded per emit, not stored as manager-level mutable state.

## Stop Conditions

Expand All @@ -191,6 +191,31 @@ The built-ins mirror the TypeScript package and OR together when provided as a l

When a stop condition fires while the model is still emitting tool calls, `call_model` makes one more turn with `tool_choice="none"` by default (tools stay in the request so the prompt-cache prefix survives) so the run ends with a natural-language answer. `allow_final_response` tunes this: `True` or omitted appends `DEFAULT_FINAL_RESPONSE_DIRECTIVE` as a user message, a non-empty string replaces the wording, `""` appends nothing, and `False` disables the extra turn entirely.

## Doom-Loop Detection

Opt in with `doom_loop=True` (defaults) or a config dict to catch a run that keeps spending without progress: the same tool call re-issued with identical arguments (including empty and invalid-JSON calls), repeated server-tool calls, or repeated text. Detection is deterministic and feeds a ladder — `observe` (emit `DoomLoopDetected`), `steer` (inject guidance as a user message before the next request), `escalate` (one-turn `model` swap and/or forced `openrouter:advisor` consult, capped by `max_escalations`), `block` (refuse the call with an error output, before `PreToolUse`), `stop` (halt before any further model request; `SessionEnd` reason `"doom_loop"`). Defaults: observe at 2, block at 3, stop at 6.

```python
result = call_model(client, {
"model": "openai/gpt-4o-mini",
"input": "Find it.",
"tools": [tool(name="web_search", input_schema=Query, execute=search,
loop_key=lambda args: args["query"].strip().lower())],
"doom_loop": {"ladder": {"steer": 2, "block": 3, "stop": 5}},
})
verdict = await result.get_doom_loop_verdict() # the stop verdict, or None
```

`loop_key` declares a tool's call identity (`False` exempts the tool; a callable returning `None` exempts that call). A `DoomLoopDetected` handler can return `{"override_action": ...}`. Detector state persists in `ConversationState.doom_loop`, so a resumed doom loop is still a doom loop.

## Usage, Tool Sets, and Tool Choice

- `await result.get_usage()` returns aggregate token/cost totals across every model call the run made (same shape as `SessionEnd`'s `total_usage`); it never raises.
- `active_tools=[...]` on `call_model` filters which client tools are sent to (and executable by) the model; server tools always pass. `create_tool_set(tools)` builds declarative `activate` / `deactivate` / `activate_when` / `deactivate_when` rules, and `tool_set.resolve().call_model` is a `{"tools", "active_tools"}` mapping to spread into the request.
- A forced `tool_choice` (`"required"`, a specific tool, or `allowed_tools` with `mode: "required"`) relaxes to `auto` after it produces a tool call, so the model can answer in text. Tools can change the next turn's `tool_choice` via `next_turn_params={"tool_choice": fn}`.
- `tool(..., strict=True)` passes `strict` through to the wire definition; `tool(..., wire_input_schema={...})` sends a caller-owned JSON Schema instead of the generated one; `server_tool(config, {"id": ...})` overrides the default `server:<type>` id.
- A response truncated at `max_output_tokens` never executes its (cut-off) tool calls.

## Format Compatibility

Use `from_claude_messages` / `to_claude_message` for Anthropic-style messages and `from_chat_messages` / `to_chat_message` for OpenAI Chat-style messages. Content that cannot be represented directly is carried as `unsupported_content` instead of being silently discarded.
Expand Down
136 changes: 136 additions & 0 deletions src/openrouter_agent/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@
)
from .hooks_manager import HooksManager
from .hooks_schemas import (
DoomLoopDetectedPayload,
DoomLoopDetectedResult,
HookDefinition,
HookName,
HookRegistry,
Expand Down Expand Up @@ -222,6 +224,74 @@
tool_has_approval_configured,
)
from .turn_context import build_turn_context, normalize_input_to_array
from .doom_loop import (
DEFAULT_DOOM_LOOP_LADDER,
DEFAULT_MAX_ESCALATIONS,
MAX_CANONICALIZE_DEPTH,
DoomLoopAction,
DoomLoopCallRecord,
DoomLoopConfig,
DoomLoopDetectorKind,
DoomLoopEscalationConfig,
DoomLoopLadder,
DoomLoopMonitor,
DoomLoopOption,
DoomLoopRoundCall,
DoomLoopSerializedState,
DoomLoopStreak,
DoomLoopTextOptions,
DoomLoopVerdict,
LoopKeyResolution,
ResolvedDoomLoopConfig,
ResolvedEscalationConfig,
TextRepetitionResult,
canonicalize_key_material,
detect_text_repetition,
fingerprint_key_material,
fingerprint_tool_call,
resolve_doom_loop_option,
resolve_ladder_action,
resolve_loop_key_material,
)
from .tool_set import (
TOOL_SET_SNAPSHOT,
ActivationInput,
ActivationPredicate,
ConditionalMode,
Directive,
ResolvedToolSnapshot,
SituationConditionalRule,
SituationConditionalRuleObject,
SituationConfig,
StatusReason,
ToolSet,
ToolStatusEntry,
create_tool_set,
get_server_tool_id,
get_tool_id,
strip_tool_set_snapshot_metadata,
)
from .user_agent import (
AGENT_PACKAGE_NAME,
AGENT_USER_AGENT,
PACKAGE_VERSION,
SDK_USER_AGENT,
agent_user_agent_token,
append_agent_user_agent,
build_agent_user_agent,
get_package_version,
)
from .reusable_stream import (
BUFFER_COMPACTION_MIN_HEAD,
ReplayConsumer,
ReusableReadableStream,
StreamReplay,
)
from .turn_context import is_input_item, output_to_input_items
from .item_types import NewAssistantMessageItem, NewDeveloperMessageItem, NewSystemMessageItem
from .chat_compat import ChatMessageInputItem
from .anthropic_compat import ClaudeMessageInputItem
from .stream_transformers import is_truncated_at_max_output_tokens, response_has_tool_calls

RequestOptions: TypeAlias = Mapping[str, Any]

Expand Down Expand Up @@ -462,4 +532,70 @@ def before_create_request(self, hook_ctx: BeforeCreateRequestContext, request: A
"tool_has_approval_configured",
"tool_requires_approval",
"update_state",
"DEFAULT_DOOM_LOOP_LADDER",
"DEFAULT_MAX_ESCALATIONS",
"MAX_CANONICALIZE_DEPTH",
"DoomLoopAction",
"DoomLoopCallRecord",
"DoomLoopConfig",
"DoomLoopDetectedPayload",
"DoomLoopDetectedResult",
"DoomLoopDetectorKind",
"DoomLoopEscalationConfig",
"DoomLoopLadder",
"DoomLoopMonitor",
"DoomLoopOption",
"DoomLoopRoundCall",
"DoomLoopSerializedState",
"DoomLoopStreak",
"DoomLoopTextOptions",
"DoomLoopVerdict",
"LoopKeyResolution",
"ResolvedDoomLoopConfig",
"ResolvedEscalationConfig",
"TextRepetitionResult",
"canonicalize_key_material",
"detect_text_repetition",
"fingerprint_key_material",
"fingerprint_tool_call",
"resolve_doom_loop_option",
"resolve_ladder_action",
"resolve_loop_key_material",
"TOOL_SET_SNAPSHOT",
"ActivationInput",
"ActivationPredicate",
"ConditionalMode",
"Directive",
"ResolvedToolSnapshot",
"SituationConditionalRule",
"SituationConditionalRuleObject",
"SituationConfig",
"StatusReason",
"ToolSet",
"ToolStatusEntry",
"create_tool_set",
"get_server_tool_id",
"get_tool_id",
"strip_tool_set_snapshot_metadata",
"AGENT_PACKAGE_NAME",
"AGENT_USER_AGENT",
"PACKAGE_VERSION",
"SDK_USER_AGENT",
"agent_user_agent_token",
"append_agent_user_agent",
"build_agent_user_agent",
"get_package_version",
"BUFFER_COMPACTION_MIN_HEAD",
"ReplayConsumer",
"ReusableReadableStream",
"StreamReplay",
"is_input_item",
"output_to_input_items",
"NewAssistantMessageItem",
"NewDeveloperMessageItem",
"NewSystemMessageItem",
"ChatMessageInputItem",
"ClaudeMessageInputItem",
"is_truncated_at_max_output_tokens",
"response_has_tool_calls",
]
9 changes: 7 additions & 2 deletions src/openrouter_agent/anthropic_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@

from ._utils import get_field, json_dumps, json_loads_maybe

ClaudeMessageInputItem = Dict[str, Any]
"""An OpenResponses input item emitted by `from_claude_messages`: a user or
assistant message item, ``function_call``, ``function_call_output``, or
``image_generation_call`` item."""


def _unsupported(original_type: str, data: Any, reason: str) -> Dict[str, Any]:
return {"original_type": original_type, "data": data, "reason": reason}


def from_claude_messages(messages: Sequence[Mapping[str, Any]]) -> List[Dict[str, Any]]:
output: List[Dict[str, Any]] = []
def from_claude_messages(messages: Sequence[Mapping[str, Any]]) -> List[ClaudeMessageInputItem]:
output: List[ClaudeMessageInputItem] = []
for message in messages:
role = str(message.get("role", "user"))
content = message.get("content", "")
Expand Down
6 changes: 6 additions & 0 deletions src/openrouter_agent/async_params.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ class CallModelInput(TypedDict, total=False):
model: str
input: Any
tools: Sequence[Any]
active_tools: Sequence[str]
stream_replay: str
stop_when: Any
state: Any
require_approval: Any
Expand All @@ -21,6 +23,7 @@ class CallModelInput(TypedDict, total=False):
allow_final_response: Any
strict_final_response: bool
hooks: Any
doom_loop: Any


CallModelInputWithState = CallModelInput
Expand All @@ -34,6 +37,8 @@ class ResolvedCallModelInput(TypedDict, total=False):


_EXCLUDED = {
"active_tools",
"stream_replay",
"stop_when",
"state",
"require_approval",
Expand All @@ -46,6 +51,7 @@ class ResolvedCallModelInput(TypedDict, total=False):
"allow_final_response",
"strict_final_response",
"hooks",
"doom_loop",
}


Expand Down
25 changes: 23 additions & 2 deletions src/openrouter_agent/call_model.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,36 @@
from __future__ import annotations

from typing import Any, Mapping, Optional
from typing import Any, Dict, Mapping, Optional

from .hooks_resolve import resolve_hooks
from .model_result import ModelResult
from .tool_executor import convert_tools_to_api_format


def call_model(client: Any, request: Mapping[str, Any], options: Optional[Mapping[str, Any]] = None) -> ModelResult:
from .tool_set import strip_tool_set_snapshot_metadata
from .tool_types import is_server_tool

tools = request.get("tools")
final_request = dict(request)
active_tools = request.get("active_tools")
# Narrow tools to the active subset (if provided) before API conversion
# and before they are registered for execution, so the model cannot call
# filtered tools and the executor does not carry orphaned definitions.
# Server tools always pass the filter.
if active_tools is not None and tools is not None:
active_set = set(active_tools)
tools = [t for t in tools if is_server_tool(t) or t.get("function", {}).get("name") in active_set]
# Collapse a filtered-to-empty (or explicitly empty) tools list to None so
# the outbound `tools` key is omitted entirely instead of sending
# `tools: []` -- several providers reject an empty array outright.
if tools is not None and len(tools) == 0:
tools = None
final_request: Dict[Any, Any] = dict(request)
strip_tool_set_snapshot_metadata(final_request)
for key in (
"tools",
"active_tools",
"stream_replay",
"stop_when",
"state",
"require_approval",
Expand All @@ -24,6 +43,7 @@ def call_model(client: Any, request: Mapping[str, Any], options: Optional[Mappin
"allow_final_response",
"strict_final_response",
"hooks",
"doom_loop",
):
final_request.pop(key, None)
if tools is not None:
Expand All @@ -50,5 +70,6 @@ def call_model(client: Any, request: Mapping[str, Any], options: Optional[Mappin
"allow_final_response": request.get("allow_final_response"),
"strict_final_response": request.get("strict_final_response"),
"hooks": resolve_hooks(request.get("hooks")),
"doom_loop": request.get("doom_loop"),
}
)
51 changes: 47 additions & 4 deletions src/openrouter_agent/chat_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@
from ._utils import get_field
from .stream_transformers import extract_responses_message_from_response

ChatMessageInputItem = Dict[str, Any]
"""An OpenResponses input item emitted by `from_chat_messages`: a message item
(user / system / assistant / developer), a ``function_call`` item, or a
``function_call_output`` item."""


def _content_to_string(content: Any) -> str:
if content is None:
Expand All @@ -15,8 +20,18 @@ def _content_to_string(content: Any) -> str:
return json.dumps(content, separators=(",", ":"))


def from_chat_messages(messages: Sequence[Mapping[str, Any]]) -> List[Dict[str, Any]]:
output: List[Dict[str, Any]] = []
def _tool_call_arguments(arguments: Any) -> str:
# Chat-format arguments are already a JSON string and are forwarded as-is
# (never re-stringified). A pre-parsed mapping is serialized once.
if arguments is None:
return "{}"
if isinstance(arguments, str):
return arguments
return json.dumps(arguments, separators=(",", ":"))


def from_chat_messages(messages: Sequence[Mapping[str, Any]]) -> List[ChatMessageInputItem]:
output: List[ChatMessageInputItem] = []
for msg in messages:
role = msg.get("role")
if role == "tool":
Expand All @@ -27,8 +42,36 @@ def from_chat_messages(messages: Sequence[Mapping[str, Any]]) -> List[Dict[str,
"output": _content_to_string(msg.get("content")),
}
)
else:
output.append({"type": "message", "role": role, "content": _content_to_string(msg.get("content"))})
continue

content = _content_to_string(msg.get("content"))
if role == "assistant":
tool_calls = list(msg.get("tool_calls") or msg.get("toolCalls") or [])

# Skip the message item only when there is no content AND there are
# tool calls to emit in its place. A content-less assistant message
# with no tool calls still round-trips as an empty message.
if content or not tool_calls:
output.append({"type": "message", "role": role, "content": content})

# One function_call item per tool call.
for tool_call in tool_calls:
call_id = get_field(tool_call, "id")
function = get_field(tool_call, "function", {}) or {}
output.append(
{
"type": "function_call",
"callId": call_id,
"id": call_id,
"name": get_field(function, "name"),
"arguments": _tool_call_arguments(get_field(function, "arguments")),
"status": "completed",
}
)
continue

# System, user, developer messages
output.append({"type": "message", "role": role, "content": content})
return output


Expand Down
Loading
Loading