Paper title
Steer, Don't Solve: Training Small Critic Models for Large Code Agents
Short name
Steer, Don't Solve
Authors
Shubham Gandhi, Yiqing Xie, Atharva Naik, Ruichen Zhu, Carolyn Rose
Publication month
2026-06
Venue
arXiv preprint
Category (can select multiple)
inference_scaling, sft
arXiv link
https://arxiv.org/abs/2606.21811
GitHub repository link
https://github.com/shubhamrgandhi/critic-training
HuggingFace link
https://huggingface.co/code-critic-model
Abstract
Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While coding agents are optimized for the joint capabilities, individual capabilities such as high-level planning may have different optima and remain a major bottleneck. To address this challenge, we train a separate critic model that is specialized in high-level planning to steer the coding agent in inference. We construct SFT and DPO data to train the critic model to identify errors made by the coding agent and provide correct and clear high-level guidance without generating concrete actions. Experiments show that our fine-tuned 4B and 8B critic models significantly improve the performance of 6 larger coding agents (e.g., improving the resolved rates of GLM-4.7-Flash-30B-A3B and GPT-OSS-120B by 16.0% and 14.4% on SWE-Bench Verified). The critic model also reduces the total inference costs for some coding agents by solving tasks in fewer steps (e.g., reducing the per-example inference cost for GPT-OSS-20B from $0.07 to $0.03). Code: https://github.com/shubhamrgandhi/critic-training
Submission checklist
Additional notes
We train separate 4B and 8B critic models (SFT, then DPO) that read the coding agent's trajectory every k steps, identify errors, and return high-level guidance without generating actions, so the primary category is Inference-Time Scaling, next to SWE-PRM and Agentic Rubrics; the SFT category covers how the critic itself is trained. Open weights and training data are on HuggingFace. I am the first author; happy to open a PR against the YAML files instead if you prefer.
Paper title
Steer, Don't Solve: Training Small Critic Models for Large Code Agents
Short name
Steer, Don't Solve
Authors
Shubham Gandhi, Yiqing Xie, Atharva Naik, Ruichen Zhu, Carolyn Rose
Publication month
2026-06
Venue
arXiv preprint
Category (can select multiple)
inference_scaling, sft
arXiv link
https://arxiv.org/abs/2606.21811
GitHub repository link
https://github.com/shubhamrgandhi/critic-training
HuggingFace link
https://huggingface.co/code-critic-model
Abstract
Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While coding agents are optimized for the joint capabilities, individual capabilities such as high-level planning may have different optima and remain a major bottleneck. To address this challenge, we train a separate critic model that is specialized in high-level planning to steer the coding agent in inference. We construct SFT and DPO data to train the critic model to identify errors made by the coding agent and provide correct and clear high-level guidance without generating concrete actions. Experiments show that our fine-tuned 4B and 8B critic models significantly improve the performance of 6 larger coding agents (e.g., improving the resolved rates of GLM-4.7-Flash-30B-A3B and GPT-OSS-120B by 16.0% and 14.4% on SWE-Bench Verified). The critic model also reduces the total inference costs for some coding agents by solving tasks in fewer steps (e.g., reducing the per-example inference cost for GPT-OSS-20B from $0.07 to $0.03). Code: https://github.com/shubhamrgandhi/critic-training
Submission checklist
Additional notes
We train separate 4B and 8B critic models (SFT, then DPO) that read the coding agent's trajectory every k steps, identify errors, and return high-level guidance without generating actions, so the primary category is Inference-Time Scaling, next to SWE-PRM and Agentic Rubrics; the SFT category covers how the critic itself is trained. Open weights and training data are on HuggingFace. I am the first author; happy to open a PR against the YAML files instead if you prefer.