10 papers
Structured Thoughts For Improved Reasoning And Context Pruning
Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu +5
Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured T…
On the Policy Gradient Foundations of Group Relative Policy Optimization: Credit Assignment, Gradient Sparsity, and Rank Collapse
Amritansh Mishra, Supriyo Chakraborty, Berkcan Kapusuzoglu
Group Relative Policy Optimization (GRPO) eliminates the learned critic in PPO by using the mean reward of grouped rollouts as a baseline. We provide a rigorous derivation of GRPO…
Know Thy Reasoner: Not All Language Models Explore Alike
Moulik Choraria, Argyrios Gerogiannis, Anirban Das +4
Compute scaling for LLM reasoning trades off exploring solution approaches (\emph{breadth}) against refining promising ones (\emph{depth}), yet why a given trade-off works, and why…
A History-Aware Visually Grounded Critic for Computer Use Agents
Jaewoo Lee, Zaid Khan, Archiki Prasad +7
Various test-time interventions for Computer Use Agents (CUAs), including critic models, have been developed to improve performance through pre-execution action evaluation in compl…
Critique-Guided Distillation for Robust Reasoning via Refinement
Berkcan Kapusuzoglu, Supriyo Chakraborty, Zain Sarwar +2
Supervised fine-tuning with expert demonstrations often produces models that imitate outputs without internalizing the reasoning processes needed for robust generalization. While c…
CoT-Guard: Small Models for Strong Monitoring
Nirav Diwan, Han Wang, Berkcan Kapusuzoglu +6
Monitoring the chain-of-thought (CoT) of reasoning models is a promising approach for detecting covert misbehavior (i.e., hidden objectives) in code generation tasks. While large m…