5 citations · 13 across the 9 of their papers we have counts for
9 papers
What Really Matters in Matrix-Whitening Optimizers?
Kevin Frans, Pieter Abbeel, Sergey Levine
A range of recent optimizers have emerged that approximate the same "matrix-whitening" transformation in various ways. In this work, we systematically deconstruct such optimizers,…
A Stable Whitening Optimizer for Efficient Neural Network Training
Kevin Frans, Sergey Levine, Pieter Abbeel
In this work, we take an experimentally grounded look at neural network optimization. Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resu…
Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding
Joshua Jones, Oier Mees, Carmelo Sferrazza +3
Interacting with the world is a multi-sensory experience: achieving effective general-purpose interaction requires making use of all available modalities -- including vision, touch…
Cliqueformer: Model-Based Optimization with Structured Transformers
Jakub Grudzien Kuba, Pieter Abbeel, Sergey Levine
Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based…
One Step Diffusion via Shortcut Models
Kevin Frans, Danijar Hafner, Sergey Levine +1
Diffusion models and flow-matching models have enabled generating diverse and realistic images by learning to transfer noise to data. However, sampling from these models involves i…
MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting
Fangchen Liu, Kuan Fang, Pieter Abbeel +1
Open-world generalization requires robotic systems to have a profound understanding of the physical world and the user command to solve diverse and complex tasks. While the recent…
Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
Zhongyu Li, Xue Bin Peng, Pieter Abbeel +3
This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single l…
FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning
Jianlan Luo, Charles Xu, Fangchen Liu +5
In this paper, we propose a real-world benchmark for studying robotic learning in the context of functional manipulation: a robot needs to accomplish complex long-horizon behaviors…
Functional Graphical Models: Structure Enables Offline Data-Driven Optimization
Jakub Grudzien Kuba, Masatoshi Uehara, Pieter Abbeel +1
While machine learning models are typically trained to solve prediction problems, we might often want to use them for optimization problems. For example, given a dataset of protein…