5 papers
AsyncWebRL: Efficient Asynchronous Reinforcement Learning for Multi-Step Visual Web Agents
Hao Bai, Rui Yang, Chenlu Ye +3
Training vision-language web agents with multi-step RL is compute-intensive, with two dominant forms of inefficiency: idle GPUs in synchronous RL, and trajectories that use more st…
What Does Flow Matching Bring To TD Learning?
Bhavya Agrawalla, Michal Nauman, Aviral Kumar
Recent work shows that flow matching can be effective for scalar Q-value function estimation in reinforcement learning (RL), but it remains unclear why or how this approach differs…
floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
Bhavya Agrawalla, Michal Nauman, Khush Agrawal +1
A hallmark of modern large-scale machine learning techniques is the use of training objectives that provide dense supervision to intermediate computations, such as teacher forcing…
Compute-Optimal Scaling for Value-Based Deep RL
Preston Fu, Oleh Rybkin, Zhiyuan Zhou +4
As models grow larger and training them becomes expensive, it becomes increasingly important to scale training recipes not just to larger models and more data, but to do so in a co…
Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners
Michal Nauman, Marek Cygan, Carmelo Sferrazza +2
Recent advances in language modeling and vision stem from training large models on diverse, multi-task data. This paradigm has had limited impact in value-based reinforcement learn…