6 papers
The Low-Rank Structure of VLA Reinforcement Learning
Minjae Oh, Yoonah Park, Jongwon Lim +1
Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models, yet how RL reshapes these policies remains poorly understood. We find that RL ac…
SHAPE of Thinking: Mathematical Heuristics for Analyzing and Improving LLM Reasoning
Jonghyun Song, Sangjun Song, Minjae Oh +3
Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet final-answer accuracy reveals little about the strategies they use. We introduce \…
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
Yunho Choi, Jongwon Lim, Woojin Ahn +3
Reinforcement learning with verifiable rewards (RLVR) for Large Reasoning Models rests on variance reduction, which requires both a reliable baseline and high prompt diversity with…
KL for a KL: On-Policy Distillation with Control Variate Baseline
Minjae Oh, Sangjun Song, Gyubin Choi +2
On-Policy Distillation (OPD) has emerged as a dominant post-training paradigm for large language models, especially for reasoning domains. However, OPD remains unstable in practice…
ThinkBrake: Efficient Reasoning via Log-Probability Margin Guided Decoding
Sangjun Song, Minjae Oh, Seungkyu Lee +2
Large Reasoning Models (LRMs) allocate substantial inference-time compute to Chain-of-Thought (CoT) reasoning, improving performance on mathematics, scientific QA, and tool usage.…
Future Policy Approximation for Offline Reinforcement Learning in LLM Reasoning
Minjae Oh, Yunho Choi, Dongmin Choi +1
Reinforcement learning (RL) has emerged as a key driver of post-training for complex reasoning in large language models (LLMs), yet online RL introduces substantial instability and…