5 papers
IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
Zhoujun Cheng, Yutao Xie, Yuxiao Qu +12
While scaling laws guide compute allocation for LLM pre-training, analogous prescriptions for reinforcement learning (RL) post-training of large language models (LLMs) remain poorl…
SAINT: Attention-Based Policies for Discrete Combinatorial Action Spaces
Matthew Landers, Taylor W. Killian, Thomas Hartvigsen +1
The combinatorial structure of many real-world action spaces leads to exponential growth in the number of possible actions, limiting the effectiveness of conventional reinforcement…
Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initialization
Matthew Landers, Taylor W. Killian, Thomas Hartvigsen +1
Reinforcement learning in discrete combinatorial action spaces requires searching over exponentially many joint actions to simultaneously select multiple sub-actions that form cohe…
BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spaces
Matthew Landers, Taylor W. Killian, Hugo Barnes +2
Offline reinforcement learning in high-dimensional, discrete action spaces is challenging due to the exponential scaling of the joint action space with the number of sub-actions an…
Robust Autonomy Emerges from Self-Play
Marco Cusumano-Towner, David Hafner, Alex Hertzberg +9
Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust an…