Publications (30)
Learning by Playing - Solving Sparse Reward Tasks from Scratch
Martin Riedmiller, Roland Hafner, Thomas Lampe +6
We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch -…
Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion
Roland Hafner, Tim Hertweck, Philipp Klöppner +6
Modern Reinforcement Learning (RL) algorithms promise to solve difficult motor control problems directly from raw sensory inputs. Their attraction is due in part to the fact that t…
Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds
Michael Neunert, Markus Stäuble, Markus Giftthaler +5
In this work we present a whole-body Nonlinear Model Predictive Control approach for Rigid Body Systems subject to contacts. We use a full dynamic system model which also includes…
Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie +115
Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as…
Mastering Stacking of Diverse Shapes with Large-Scale Iterative Reinforcement Learning on Real Robots
Thomas Lampe, Abbas Abdolmaleki, Sarah Bechtle +12
Reinforcement learning solely from an agent's self-generated data is often believed to be infeasible for learning on real robots, due to the amount of data needed. However, if done…
Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes
Alex X. Lee, Coline Devin, Yuxiang Zhou +18
We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategie…
Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
Michael Neunert, Abbas Abdolmaleki, Markus Wulfmeier +7
Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision…
Simultaneously Learning Vision and Feature-based Control Policies for Real-world Ball-in-a-Cup
Devin Schwab, Tobias Springenberg, Murilo F. Martins +7
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary…
Fast Trajectory Optimization for Legged Robots using Vertex-based ZMP Constraints
Alexander W Winkler, Farbod Farshidian, Diego Pardo +2
This paper combines the fast Zero-Moment-Point (ZMP) approaches that work well in practice with the broader range of capabilities of a Trajectory Optimization formulation, by optim…
Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer
Gemini Robotics Team, Abbas Abdolmaleki, Saminda Abeyruwan +169
General-purpose robots need a deep understanding of the physical world, advanced reasoning, and general and dexterous control. This report introduces the latest generation of the G…
A Family of Iterative Gauss-Newton Shooting Methods for Nonlinear Optimal Control
Markus Giftthaler, Michael Neunert, Markus Stäuble +2
This paper introduces a family of iterative algorithms for unconstrained nonlinear optimal control. We generalize the well-known iLQR algorithm to different multiple-shooting varia…
Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning
Noah Y. Siegel, Jost Tobias Springenberg, Felix Berkenkamp +6
Off-policy reinforcement learning algorithms promise to be applicable in settings where only a fixed data-set (batch) of environment interactions is available and no new experience…
Projection based whole body motion planning for legged robots
Diego Pardo, Michael Neunert, Alexander W. Winkler +1
In this paper we present a new approach for dynamic motion planning for legged robots. We formulate a trajectory optimization problem based on a compact form of the robot dynamics.…
SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Exploration
Giulia Vezzani, Dhruva Tirumala, Markus Wulfmeier +13
The ability to effectively reuse prior knowledge is a key requirement when building general and flexible Reinforcement Learning (RL) agents. Skill reuse is one of the most common a…
Automatic Differentiation of Rigid Body Dynamics for Optimal Control and Estimation
Markus Giftthaler, Michael Neunert, Markus Stäuble +3
Many algorithms for control, optimization and estimation in robotics depend on derivatives of the underlying system dynamics, e.g. to compute linearizations, sensitivities or gradi…
An Efficient Optimal Planning and Control Framework For Quadrupedal Locomotion
Farbod Farshidian, Michael Neunert, Alexander W. Winkler +2
In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem…
Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors
Steven Bohez, Saran Tunyasuvunakool, Philemon Brakel +18
We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitati…
"What, not how": Solving an under-actuated insertion task from scratch
Giulia Vezzani, Michael Neunert, Markus Wulfmeier +7
Robot manipulation requires a complex set of skills that need to be carefully combined and coordinated to solve a task. Yet, most ReinforcementLearning (RL) approaches in robotics…
Compositional Transfer in Hierarchical Reinforcement Learning
Markus Wulfmeier, Abbas Abdolmaleki, Roland Hafner +7
The successful application of general reinforcement learning algorithms to real-world robotics applications is often limited by their high data requirements. We introduce Regulariz…
A Distributional View on Multi-Objective Policy Optimization
Abbas Abdolmaleki, Sandy H. Huang, Leonard Hasenclever +7
Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for…
Barkour: Benchmarking Animal-level Agility with Quadruped Robots
Ken Caluwaerts, Atil Iscen, J. Chase Kew +41
Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biol…
Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning
Mohak Bhardwaj, Thomas Lampe, Michael Neunert +6
Recent advances in real-world applications of reinforcement learning (RL) have relied on the ability to accurately simulate systems at scale. However, domains such as fluid dynamic…
Value constrained model-free continuous control
Steven Bohez, Abbas Abdolmaleki, Michael Neunert +3
The naive application of Reinforcement Learning algorithms to continuous control problems -- such as locomotion and manipulation -- often results in policies which rely on high-amp…
Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Models
Arunkumar Byravan, Jost Tobias Springenberg, Abbas Abdolmaleki +6
Humans are masters at quickly learning many complex tasks, relying on an approximate understanding of the dynamics of their environments. In much the same way, we would like our le…
Data-efficient Hindsight Off-policy Option Learning
Markus Wulfmeier, Dushyant Rao, Roland Hafner +8
We introduce Hindsight Off-policy Options (HO2), a data-efficient option learning algorithm. Given any trajectory, HO2 infers likely option choices and backpropagates through the d…
Towards practical reinforcement learning for tokamak magnetic control
Brendan D. Tracey, Andrea Michi, Yuri Chervonyi +15
Reinforcement learning (RL) has shown promising results for real-time control systems, including the domain of plasma magnetic control. However, there are still significant drawbac…
Trajectory Optimization Through Contacts and Automatic Gait Discovery for Quadrupeds
Michael Neunert, Farbod Farshidian, Alexander W. Winkler +1
In this work we present a trajectory Optimization framework for whole-body motion planning through contacts. We demonstrate how the proposed approach can be applied to automaticall…
Evaluating direct transcription and nonlinear optimization methods for robot motion planning
Diego Pardo, Lukas Möller, Michael Neunert +2
This paper studies existing direct transcription methods for trajectory optimization applied to robot motion planning. There are diverse alternatives for the implementation of dire…
An Open Source, Fiducial Based, Visual-Inertial Motion Capture System
Michael Neunert, Michael Bloesch, Jonas Buchli
Many robotic tasks rely on the accurate localization of moving objects within a given workspace. This information about the objects' poses and velocities are used for control,motio…
The Control Toolbox - An Open-Source C++ Library for Robotics, Optimal and Model Predictive Control
Markus Giftthaler, Michael Neunert, Markus Stäuble +1
We introduce the Control Toolbox (CT), an open-source C++ library for efficient modeling, control, estimation, trajectory optimization and Model Predictive Control. The CT is appli…