Publications (40)
Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie +115
Sequence-of-Constraints MPC: Reactive Timing-Optimal Control of Sequential Manipulation
Marc Toussaint, Jason Harris, Jung-Su Ha +2
: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities
Physical Intelligence, Bo Ai, Ali Amin +85
RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation
Soroush Nasiriany, Sean Kirmani, Tianli Ding +5
Reinforcement Learning with Neural Radiance Fields
Danny Driess, Ingmar Schubert, Pete Florence +2
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Anthony Brohan, Noah Brown, Justice Carbajal +51
Probabilistic Framework for Constrained Manipulations and Task and Motion Planning under Uncertainty
Jung-Su Ha, Danny Driess, Marc Toussaint
: A Vision-Language-Action Flow Model for General Robot Control
Kevin Black, Noah Brown, Danny Driess +21
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Vidhi Jain, Maria Attarian, Nikhil J Joshi +10
Deep 6-DoF Tracking of Unknown Objects for Reactive Grasping
Marc Tuscher, Julian Hörz, Danny Driess +1
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
Boyuan Chen, Zhuo Xu, Sean Kirmani +6
Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning
Danny Driess, Jung-Su Ha, Marc Toussaint +1
Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models
Lucy Xiaoyang Shi, Brian Ichter, Michael Equi +12
Towards Generalist Biomedical AI
Tao Tu, Shekoofeh Azizi, Danny Driess +29
Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image
Danny Driess, Jung-Su Ha, Marc Toussaint
Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better
Danny Driess, Jost Tobias Springenberg, Brian Ichter +8
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Soroush Nasiriany, Fei Xia, Wenhao Yu +20
: a Vision-Language-Action Model with Open-World Generalization
Physical Intelligence, Kevin Black, Noah Brown +33
Large Language Models as General Pattern Machines
Suvir Mirchandani, Fei Xia, Pete Florence +6
: a VLA That Learns From Experience
Physical Intelligence, Ali Amin, Raichelle Aniceto +53
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Karl Pertsch, Kyle Stachowicz, Brian Ichter +6
Long-Horizon Multi-Robot Rearrangement Planning for Construction Assembly
Valentin Noah Hartmann, Andreas Orthey, Danny Driess +2
ALOHA Unleashed: A Simple Recipe for Robot Dexterity
Tony Z. Zhao, Jonathan Tompson, Danny Driess +4
PaLM-E: An Embodied Multimodal Language Model
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi +19
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
Training Strategies for Efficient Embodied Reasoning
William Chen, Suneel Belkhale, Suvir Mirchandani +4
Learning Feasibility of Factored Nonlinear Programs in Robotic Manipulation Planning
Joaquim Ortiz-Haro, Jung-Su Ha, Danny Driess +2
Robust Task and Motion Planning for Long-Horizon Architectural Construction Planning
Valentin N. Hartmann, Ozgur S. Oguz, Danny Driess +2
Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Input
Jung-Su Ha, Danny Driess, Marc Toussaint
Direct Motion Models for Assessing Generated Videos
Kelsey Allen, Carl Doersch, Guangyao Zhou +9
Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control
William Chen, Jagdeep Singh Bhatia, Catherine Glossop +6
Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents
Wenlong Huang, Fei Xia, Dhruv Shah +8
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Vision Language Models are In-Context Value Learners
Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid +15
Learning Multi-Object Dynamics with Compositional Neural Radiance Fields
Danny Driess, Zhiao Huang, Yunzhu Li +2
Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics
Ingmar Schubert, Danny Driess, Ozgur S. Oguz +1
FC: Feasibility-Based Control Chain Coordination
Jason Harris, Danny Driess, Marc Toussaint
MEM: Multi-Scale Embodied Memory for Vision Language Action Models
Marcel Torne, Karl Pertsch, Homer Walke +14
Describing Physics For Physical Reasoning: Force-based Sequential Manipulation Planning
Marc Toussaint, Jung-Su Ha, Danny Driess
Foundation Models in Robotics: Applications, Challenges, and the Future
Roya Firoozi, Johnathan Tucker, Stephen Tian +12