11 papers
CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data
Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru +3
Evaluations should do more than measure a models current performance. They should tell us what to fix for the next model iteration and provide a way to generate targeted post train…
Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers
MohammadHossein Rezaei, Anas Mahmoud, Zihao Wang +6
Rubrics have emerged as an alternative to RLVR in open-ended domains where a single ground-truth final answer is not available. Existing rubric-based training methods rely on an LL…
From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data
Mohammad R. Rezaei, Rahul G. Krishnan
A persistent challenge in machine learning for scientific applications is jointly achieving prediction and understanding. Statistical models excel on structured data but operate as…
Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
Utkarsh Tyagi, Xingang Guo, MohammadHossein Rezaei +5
Reinforcement learning with verifiable rewards has made post-training highly effective when correctness can be checked automatically. However, many important model behaviors requir…
Reward Hacking in Rubric-Based Reinforcement Learning
Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang +3
Reinforcement learning with verifiable rewards has enabled strong post-training gains in domains such as math and coding, though many open-ended settings rely on rubric-based rewar…
SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
Mohit Raghavendra, Soham Dan, Miguel Romero Calvo +12
We introduce SWE Atlas, a benchmark suite for coding agents spanning three professional software engineering workflows: Codebase Q&A (124 tasks), Test Writing (90 tasks), and Refac…