5 papers
SHAPE of Chain-of-Thought in Math Reasoning
Jonghyun Song, Sangjun Song, Minjae Oh +3
Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their reasoning remain underexplor…
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
Yunho Choi, Jongwon Lim, Woojin Ahn +3
Reinforcement learning with verifiable rewards (RLVR) for Large Reasoning Models hinges on baseline estimation for variance reduction, but existing approaches pay a heavy price: PP…
KL for a KL: On-Policy Distillation with Control Variate Baseline
Minjae Oh, Sangjun Song, Gyubin Choi +2
On-Policy Distillation (OPD) has emerged as a dominant post-training paradigm for large language models, especially for reasoning domains. However, OPD remains unstable in practice…
ThinkBrake: Efficient Reasoning via Log-Probability Margin Guided Decoding
Sangjun Song, Minjae Oh, Seungkyu Lee +2
Large Reasoning Models (LRMs) allocate substantial inference-time compute to Chain-of-Thought (CoT) reasoning, improving performance on mathematics, scientific QA, and tool usage.…
Future Policy Approximation for Offline Reinforcement Learning in LLM Reasoning
Minjae Oh, Yunho Choi, Dongmin Choi +1
Reinforcement learning (RL) has emerged as a key driver of post-training for complex reasoning in large language models (LLMs), yet online RL introduces substantial instability and…