[MindOfqc (MindOfqc)]

Summary

ReasoningEfficiencyEvaluator handles LLM responses that are lists of messages, but reads each message's content as a string. A valid multimodal response (content: [{"type": "text", "text": ...}]) therefore:

• crashes loop detection with AttributeError: 'list' object has no attribute 'lower',
• crashes _get_call_samples with TypeError: sequence item 0: expected str instance, list found,
• and makes _analyze_reasoning_patterns count content parts (1) instead of text characters (19), which skews the pattern classification.

The fix uses the existing message_content_text helper at the three boundaries: loop detection and readable samples now use the extracted text, and response-length statistics count its characters. Raw-string responses are untouched, and missing or non-text content still falls back to the helper's existing behaviour ("" / "[multimodal content]").

Type of change

• Bug fix (non-breaking change that fixes an issue)

How Has This Been Tested?

Three cases added to lib/crewai/tests/experimental/evaluation/metrics/test_reasoning_metrics.py: a three-call trace whose response is [{"role": "assistant", "content": [{"type": "text", "text": "Review the evidence"}]}], asserted through _detect_loops, _get_call_samples and _analyze_reasoning_patterns.

Before:

AttributeError: 'list' object has no attribute 'lower'
TypeError: sequence item 0: expected str instance, list found
AssertionError: assert np.float64(1.0) == 19
3 failed

After:

7 passed   # test_reasoning_metrics.py
30 passed  # lib/crewai/tests/experimental/evaluation

ruff check on both files reports no new findings (the file keeps its 8 pre-existing ones).

Fixes #8796

Prepared with an AI coding agent.

---

Note

Low Risk
Scoped to experimental reasoning evaluation parsing; uses an existing shared helper and leaves string responses as-is.

Overview
Fixes ReasoningEfficiencyEvaluator so LLM traces whose response is a message list with structured multimodal content (e.g. [{"type": "text", "text": "..."}]) no longer break loop detection, sample rendering, or length metrics.

Loop detection, call samples, and avg_length now use message_content_text instead of treating msg["content"] as a string—avoiding crashes and counting one part instead of real text length. Plain string responses are unchanged.

Adds TestReasoningEfficiencyMultimodalResponses covering _detect_loops, _get_call_samples, and _analyze_reasoning_patterns on repeated multimodal traces.

^{Reviewed by Cursor Bugbot for commit 0edccc7. Bugbot is set up for automated code reviews on this repo. Configure here.}

[strix-security]

Strix Security Review

Warning

This pull request has 79 commits after the last Strix review (13f3585). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Fixes ReasoningEfficiencyEvaluator so that LLM responses whose message content is multimodal (a list of parts) are read through the shared message_content_text helper instead of being treated as strings, preventing crashes and giving correct length statistics.

Review summary

The PR hardens ReasoningEfficiencyEvaluator against multimodal (list) response content by routing three text-reading boundaries through the existing message_content_text helper. The change only affects loop-detection text extraction, response-length statistics, and readable call samples; the helper safely handles missing, non-list, and non-text content, and no untrusted data reaches a new dangerous sink. No security issues were found in the changed code.

_{Review 1 on this PR · 13f3585 · requested Oct 30 02:33 UTC when the PR was opened}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}