4 papers
IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
Zhoujun Cheng, Yutao Xie, Yuxiao Qu +12
While scaling laws guide compute allocation for LLM pre-training, analogous prescriptions for reinforcement learning (RL) post-training of large language models (LLMs) remain poorl…
POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
Yuxiao Qu, Amrith Setlur, Virginia Smith +2
Reinforcement learning (RL) has improved the reasoning abilities of large language models (LLMs), yet state-of-the-art methods still fail to learn on many training problems. On har…
On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift
Pratiksha Thaker, Amrith Setlur, Zhiwei Steven Wu +1
Public pretraining is a promising approach to improve differentially private model training. However, recent work has noted that many positive research results studying this paradi…
e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs
Amrith Setlur, Matthew Y. R. Yang, Charlie Snell +5
Test-time scaling offers a promising path to improve LLM reasoning by utilizing more compute at inference time; however, the true promise of this paradigm lies in extrapolation (i.…