2 citations · 4 across the 5 of their papers we have counts for
5 papers · 1 filter
What Really Matters in Matrix-Whitening Optimizers?
Kevin Frans, Pieter Abbeel, Sergey Levine
A range of recent optimizers have emerged that approximate the same "matrix-whitening" transformation in various ways. In this work, we systematically deconstruct such optimizers,…
A Stable Whitening Optimizer for Efficient Neural Network Training
Kevin Frans, Sergey Levine, Pieter Abbeel
In this work, we take an experimentally grounded look at neural network optimization. Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resu…
Cliqueformer: Model-Based Optimization with Structured Transformers
Jakub Grudzien Kuba, Pieter Abbeel, Sergey Levine
Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based…
One Step Diffusion via Shortcut Models
Kevin Frans, Danijar Hafner, Sergey Levine +1
Diffusion models and flow-matching models have enabled generating diverse and realistic images by learning to transfer noise to data. However, sampling from these models involves i…
Functional Graphical Models: Structure Enables Offline Data-Driven Optimization
Jakub Grudzien Kuba, Masatoshi Uehara, Pieter Abbeel +1
While machine learning models are typically trained to solve prediction problems, we might often want to use them for optimization problems. For example, given a dataset of protein…