Skip to content

fix(devin): rewrite the Claude Code sentences Cognition refuses (#8558) - #8703

Closed
jhl880617 wants to merge 1 commit into
lidge-jun:devfrom
jhl880617:fix/8558-claude-code-cognition-rewrites
Closed

jhl880617 wants to merge 1 commit into
lidge-jun:devfrom
jhl880617:fix/8558-claude-code-cognition-rewrites

Conversation

@jhl880617

@jhl880617 jhl880617 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Claude Code's system prompt contains three whole sentences that Cognition's content filter rejects with permission_denied. Every Claude Code turn routed to a Devin model therefore failed, while Codex worked on the same account. This adds those three sentences to COGNITION_BLOCKLIST_REWRITES as meaning-preserving rewrites.

Closes #8558.

Rebased onto dev after #8590 merged, as requested: the three entries follow the two Codex boilerplate entries, the comment paragraph is position-independent, #8590's Scope note is untouched, and the test file keeps its single streamChatEvents, type ChatHistoryItem import.

What changed

  • src/adapters/devin/cloud-direct/chat.ts: three new entries appended to COGNITION_BLOCKLIST_REWRITES, matched like the existing Codex entries (case-insensitive, flexible whitespace, optional commas). The rewrites keep the original authorization/safety meaning:
    • …Claude Agent SDK → …Claude Agent Toolkit
    • Dual-use security tools … require clear authorization context: … → Dual-purpose security tools … need clear authorization context: … (the authorization requirement and its list of contexts are unchanged)
    • Claude Code is available as a CLI … → Claude Code ships as a CLI …
  • Existing entries, including the case-sensitive Takes a task_id… one, and fix(devin): handle denied Codex boilerplate and optional history rewrites #8590's data-surface opt-in are unchanged.
  • tests/providers/devin-adapter.test.ts: three regression tests through the real request encoder, not only the regex table.

Verification

  • bun run test tests/providers/devin-adapter.test.ts → 58 pass, 0 fail (424 expect() calls). The 58 includes fix(devin): handle denied Codex boilerplate and optional history rewrites #8590's tests; the three added here are the #8558 ones.
  • bun run typecheck (bun check --no-script --noEmit) → no type errors in 2,533 files.
  • Full suite not run locally — a local tests/providers run in one process shows pre-existing cross-file interference between the Kiro and Devin files (reproduces with this patch reverted), so CI is the authority for the whole tree.

What the wire-level tests assert:

  • buildGetChatMessageRequest with all three sentences in the leading system prompt and verbatim in user text, replayed assistant history/thinking, tool-call arguments and a tool result:
    • field [codex] Add Neuralwatt effort routing #2 (the system-prompt instruction surface) decodes to the fully rewritten prompt, with surrounding text intact;
    • every field [codex] Refresh Codex cache after provider changes #3 message body is byte-identical to a control request encoded without the system prompt (only the per-request random message id is normalized), so message content, history, thinking, tool-call arguments and tool results are unchanged under the default policy;
    • the literal sentences still appear verbatim in those data fields.
  • Tool definitions: a description containing a sentence is rewritten, while the same bytes in tool-call arguments stay literal.
  • Table-level: the three originals plus lowercase / missing-comma / doubled-space variants are rewritten; benign sub-phrases (Claude Agent SDK, credential testing, VS Code, JetBrains, Claude Code is available) are left alone.

Evidence

Live observation on a Devin account, disclosed in #8558: each of the three original sentences alone returned 403 permission_denied against swe-2-flash, while the rewritten sentences (alone and combined in one prompt) returned 200; the same account served Codex turns normally. Lowercasing, dropping a comma, or doubling a space was still refused, which is why these entries match flexibly like the Codex ones.

Review readiness checklist

  • Required local validation passed; commands, results, and any full-suite exception are documented. (Targeted test file + typecheck; the full suite was not run locally — CI is the authority.)
  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge). (Rebased onto dev @ 1d10758, after fix(devin): handle denied Codex boilerplate and optional history rewrites #8590 merged as 4cbe4be.)
  • I resolved all correct Codex and CodeRabbit findings. (No actionable comments were generated.)
  • My PR is ready for review.

🤖 Generated with Claude Code

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of selected Claude Code system-prompt wording when communicating through the Devin adapter, including variations in capitalization, spacing, and punctuation.
    • Rewrites are limited to matching system-instruction and tool-description text; conversation history, tool inputs, and tool results remain unchanged.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

Hygiene

✅ Deterministic PR hygiene checks passed.

@github-actions
github-actions Bot marked this pull request as draft November 13, 2026 16:28
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Devin adapter adds three rewrites for Claude Code sentences that trigger Cognition’s blocklist. Tests cover case and whitespace variants, preservation of surrounding text, and the scope of changes to outgoing request fields.

Changes: Devin Claude Code prompt rewrites

Layer / File(s) Summary
Add and verify blocklist rewrites
src/adapters/devin/cloud-direct/chat.ts, tests/providers/devin-adapter.test.ts
The adapter documents three Claude Code sentences and adds case-insensitive rewrites with flexible whitespace and optional commas. Sanitizer tests cover sentence variants, surrounding text, and selected benign phrases.
Verify rewrites in outgoing requests
tests/providers/devin-adapter.test.ts
Wire-level tests verify rewritten system-instruction and tool-description fields. They also verify that message bodies, thinking, tool-call arguments, and tool results remain unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: Minimal · up to 37811

This change rewrites three Claude Code prompt sentences that Cognition refuses, so Claude Code requests routed to Devin should no longer be rejected. The rewrites apply only to instruction surfaces, and tests check that message bodies and tool-call arguments are unchanged. No merge-blocking risk is evident.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the primary change: rewriting three Claude Code system-prompt sentences that Cognition refuses, in the Devin adapter. The raw summary and objectives confirm that COGNITION_BLOCKLIST_RE…
Linked Issues check Passed [#8558] Required: add meaning-preserving rewrites for the three Claude Code system-prompt sentences in COGNITION_BLOCKLIST_REWRITES, use flexible matching (case, whitespace, optional commas), and add …
Out of Scope Changes check Passed The whole-PR diff against merge base 1d10758 changes two files. chat.ts adds a 10-line comment block that cites #8558 and three entries to COGNITION_BLOCKLIST_REWRITES. The existing entries, including…
Full details: Title check

Explanation

The title names the primary change: rewriting three Claude Code system-prompt sentences that Cognition refuses, in the Devin adapter. The raw summary and objectives confirm that COGNITION_BLOCKLIST_REWRITES gains three case-insensitive, whitespace-tolerant rewrites with regression tests. The title's '(#8558)' is the linked issue number, which is acceptable context. Title is specific and matches the changeset.

Full details: Linked Issues check

Explanation

[#8558] Required: add meaning-preserving rewrites for the three Claude Code system-prompt sentences in COGNITION_BLOCKLIST_REWRITES, use flexible matching (case, whitespace, optional commas), and add regression tests. Whole-PR diff against merge base 1d10758 shows all three requirements in src/adapters/devin/cloud-direct/chat.ts: (1) 'You are a Claude agent, built on Anthropic's Claude Agent SDK' maps to '...Claude Agent Toolkit'; (2) the Dual-use security tools sentence maps to 'Dual-purpose ... need clear authorization context: ...' with the same contexts list; (3) 'Claude Code is available as a CLI ...' maps to 'Claude Code ships as a CLI ...'. Each regex uses the i flag, \s+ between words, and ,? for the optional commas, matching the issue's suggested patterns. Tests in tests/providers/devin-adapter.test.ts (+167 lines) cover table-level rewrites, lowercase/missing-comma/double-space variants, sub-phrase passthrough, wire-level encoding (field #2 rewritten, #3 bodies byte-identical to a control request), and tool descriptions versus tool arguments. The issue named tests/devin-adapter.test.ts; the suite lives at tests/providers/devin-adapter.test.ts, which is an acceptable location. The live 403/200 probe results are author-reported and were not re-run. The test run and typecheck results are author-reported and were not re-run. The issue's coding requirements are met at the code level.

Full details: Out of Scope Changes check

Explanation

The whole-PR diff against merge base 1d10758 changes two files. chat.ts adds a 10-line comment block that cites #8558 and three entries to COGNITION_BLOCKLIST_REWRITES. The existing entries, including the case-sensitive TaskOutput entry, are unchanged in the hunks shown. The test file adds three #8558 tests and no unrelated changes. Each change connects directly to #8558's requested rewrites and regression tests. No unrelated refactoring or cleanup was found.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between e991b77 and 378118c.

Files selected for processing: 2

  • src/adapters/devin/cloud-direct/chat.ts
  • tests/providers/devin-adapter.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@lidge-jun

Copy link
Copy Markdown
Owner

Merge-manager note: #8590 just merged (4cbe4be) and touches the same COGNITION_BLOCKLIST_REWRITES table, comment block, import line and test insertion point, so this branch now conflicts. Please merge dev, append your three entries after #8590's two, reword the comment to "The three #8558 entries (Claude Code system prompt)…" so it doesn't depend on position, keep #8590's Scope note, and import streamChatEvents, type ChatHistoryItem once. The change itself reviewed clean (non-overlapping patterns, byte-identical data bodies, real-encoder tests). Then tick the readiness boxes on that head.

…e-jun#8558)

Claude Code's system prompt carries three whole sentences that Cognition's
content filter refuses with permission_denied, so every Claude Code turn to a
Devin model failed while Codex worked on the same account. Each sentence was
refused alone and on lowercase/whitespace/comma variants, so they are matched
like the Codex entries and reworded without losing their
authorization/safety meaning.

The regression tests build requests through buildGetChatMessageRequest, assert
the lidge-jun#2 system prompt is rewritten end to end, and assert every lidge-jun#3 message body
(user text, history, thinking, tool-call arguments, tool results) is
byte-identical to a control encoded without the system prompt. Existing table
entries and their case sensitivity are unchanged.

Rebased onto dev after lidge-jun#8590 merged: the three entries follow the two Codex
boilerplate entries, the comment no longer depends on table position, lidge-jun#8590's
Scope note is untouched, and the test file keeps its single
`streamChatEvents, type ChatHistoryItem` import.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@jhl880617
jhl880617 force-pushed the fix/8558-claude-code-cognition-rewrites branch from e991b77 to 378118c Compare November 15, 2026 15:59
@jhl880617
jhl880617 marked this pull request as ready for review November 15, 2026 15:59
@github-actions
github-actions Bot marked this pull request as draft November 15, 2026 15:59
lidge-jun added a commit that referenced this pull request Nov 17, 2026
…nclude, Command Code root schemas, JEV continuation effort (#8873)

Lane R3 of the 2026-11-17 triage round: rewrite the Claude Code sentences Cognition refuses for Devin (#8703), normalize the OpenCode Go output include (#8827), normalize root tool schemas for Command Code's Claude upstreams (#8591), and keep JEV's decided effort through tool-loop continuations (#8861, continuation fix only), with review follow-ups.

Co-authored-by: 袁威豪 <137183806+jhl880617@users.noreply.github.com>
Co-authored-by: Ingwannu <186453546+Ingwannu@users.noreply.github.com>
Co-authored-by: Mursyid <ciddhere@gmail.com>
Co-authored-by: tachyon6 <songjunjun62754@gmail.com>
@lidge-jun

Copy link
Copy Markdown
Owner

Thank you. This change was carried, with your authorship kept as a Co-authored-by trailer, in #8873, which is now merged into dev as 2590e2e (2026-11-17 triage round). Closing as superseded; see #8873 for the review follow-ups made on top.

@lidge-jun lidge-jun closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Devin via Claude Code returns 403 permission_denied: unlisted Cognition blocklist phrases in Claude Code system prompt

2 participants