20 citations · 34 across the 18 of their papers we have counts for
12 papers · 1 filter
What Really Matters in Matrix-Whitening Optimizers?
Kevin Frans, Pieter Abbeel, Sergey Levine
A range of recent optimizers have emerged that approximate the same "matrix-whitening" transformation in various ways. In this work, we systematically deconstruct such optimizers,…
Compute-Optimal Scaling for Value-Based Deep RL
Preston Fu, Oleh Rybkin, Zhiyuan Zhou +4
As models grow larger and training them becomes expensive, it becomes increasingly important to scale training recipes not just to larger models and more data, but to do so in a co…
A Stable Whitening Optimizer for Efficient Neural Network Training
Kevin Frans, Sergey Levine, Pieter Abbeel
In this work, we take an experimentally grounded look at neural network optimization. Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resu…
Diffusion Guidance Is a Controllable Policy Improvement Operator
Kevin Frans, Seohong Park, Pieter Abbeel +1
At the core of reinforcement learning is the idea of learning beyond the performance in the data. However, scaling such systems has proven notoriously tricky. In contrast, techniqu…
Value-Based Deep RL Scales Predictably
Oleh Rybkin, Michal Nauman, Preston Fu +4
Scaling data and compute is critical to the success of modern ML. However, scaling demands predictability: we want methods to not only perform well with more compute or data, but a…
Prioritized Generative Replay
Renhao Wang, Kevin Frans, Pieter Abbeel +2
Sample-efficient online reinforcement learning often uses replay buffers to store experience for reuse when updating the value function. However, uniform replay is inefficient, sin…