198 citations · 532 across the 19 of their papers we have counts for
19 papers
Splatting Physical Scenes: End-to-End Real-to-Sim from Imperfect Robot Data
Ben Moran, Mauro Comi, Arunkumar Byravan +4
Creating accurate, physical simulations directly from real-world robot motion holds great value for safe, scalable, and affordable robot learning, yet remains exceptionally challen…
Motion Tracking with Muscles: Predictive Control of a Parametric Musculoskeletal Canine Model
Vittorio La Barbera, Steven Bohez, Leonard Hasenclever +2
We introduce a novel musculoskeletal model of a dog, procedurally generated from accurate 3D muscle meshes. Accompanying this model is a motion capture-based locomotion task compat…
Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie +115
Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as…
Proc4Gem: Foundation models for physical agency through procedural generation
Yixin Lin, Jan Humplik, Sandy H. Huang +18
In robot learning, it is common to either ignore the environment semantics, focusing on tasks like whole-body control which only require reasoning about robot-environment contacts,…
Barkour: Benchmarking Animal-level Agility with Quadruped Robots
Ken Caluwaerts, Atil Iscen, J. Chase Kew +41
Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biol…
NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields
Arunkumar Byravan, Jan Humplik, Leonard Hasenclever +8
We present a system for applying sim2real approaches to "in the wild" scenes with realistic visuals, and to policies which rely on active perception using RGB cameras. Given a shor…
Forgetting and Imbalance in Robot Lifelong Learning with Off-policy Data
Wenxuan Zhou, Steven Bohez, Jan Humplik +5
Robots will experience non-stationary environment dynamics throughout their lifetime: the robot dynamics can change due to wear and tear, or its surroundings may change over time.…
Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors
Steven Bohez, Saran Tunyasuvunakool, Philemon Brakel +18
We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitati…
Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner
Philemon Brakel, Steven Bohez, Leonard Hasenclever +2
Dynamic quadruped locomotion over challenging terrains with precise foot placements is a hard problem for both optimal control methods and Reinforcement Learning (RL). Non-linear s…
dm_control: Software and Tasks for Continuous Control
Yuval Tassa, Saran Tunyasuvunakool, Alistair Muldal +8
The dm_control software package is a collection of Python libraries and task suites for reinforcement learning agents in an articulated-body simulation. A MuJoCo wrapper provides c…
Value constrained model-free continuous control
Steven Bohez, Abbas Abdolmaleki, Michael Neunert +3
The naive application of Reinforcement Learning algorithms to continuous control problems -- such as locomotion and manipulation -- often results in policies which rely on high-amp…
Relative Entropy Regularized Policy Iteration
Abbas Abdolmaleki, Jost Tobias Springenberg, Jonas Degrave +5
We present an off-policy actor-critic algorithm for Reinforcement Learning (RL) that combines ideas from gradient-free optimization via stochastic search with learned action-value…
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Jie Tan, Tingnan Zhang, Erwin Coumans +5
Designing agile locomotion for quadruped robots often requires extensive expertise and tedious manual tuning. In this paper, we present a system to automate this process by leverag…
Transfer Learning with Binary Neural Networks
Sam Leroux, Steven Bohez, Tim Verbelen +3
Previous work has shown that it is possible to train deep neural networks with low precision weights and activations. In the extreme case it is even possible to constrain the netwo…
Decoupled Learning of Environment Characteristics for Safe Exploration
Pieter Van Molle, Tim Verbelen, Steven Bohez +3
Reinforcement learning is a proven technique for an agent to learn a task. However, when learning a task using reinforcement learning, the agent cannot distinguish the characterist…
Sensor Fusion for Robot Control through Deep Reinforcement Learning
Steven Bohez, Tim Verbelen, Elias De Coninck +3
Deep reinforcement learning is becoming increasingly popular for robot control algorithms, with the aim for a robot to self-learn useful feature representations from unstructured s…
Lazy Evaluation of Convolutional Filters
Sam Leroux, Steven Bohez, Cedric De Boom +5
In this paper we propose a technique which avoids the evaluation of certain convolutional filters in a deep neural network. This allows to trade-off the accuracy of a deep neural n…
Efficiency Evaluation of Character-level RNN Training Schedules
Cedric De Boom, Sam Leroux, Steven Bohez +3
We present four training and prediction schedules from the same character-level recurrent neural network. The efficiency of these schedules is tested in terms of model effectivenes…
Learning Semantic Similarity for Very Short Texts
Cedric De Boom, Steven Van Canneyt, Steven Bohez +2
Levering data on social media, such as Twitter and Facebook, requires information retrieval algorithms to become able to relate very short text fragments to each other. Traditional…