109 citations · 273 across the 18 of their papers we have counts for
20 papers
Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer
Gemini Robotics Team, Abbas Abdolmaleki, Saminda Abeyruwan +169
General-purpose robots need a deep understanding of the physical world, advanced reasoning, and general and dexterous control. This report introduces the latest generation of the G…
Code World Models for General Game Playing
Wolfgang Lehrach, Daniel Hennes, Miguel Lazaro-Gredilla +13
Large Language Models (LLMs) reasoning abilities are increasingly being applied to classical board and card games, but the dominant approach -- involving prompting for direct move…
SAS-Prompt: Large Language Models as Numerical Optimizers for Robot Self-Improvement
Heni Ben Amor, Laura Graesser, Atil Iscen +7
We demonstrate the ability of large language models (LLMs) to perform iterative self-improvement of robot policies. An important insight of this paper is that LLMs have a built-in…
Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie +115
Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as…
Achieving Human Level Competitive Robot Table Tennis
David B. D'Ambrosio, Saminda Abeyruwan, Laura Graesser +24
Achieving human-level speed and performance on real world tasks is a north star for the robotics research community. This work takes a step towards that goal and presents the first…
Prosody for Intuitive Robotic Interface Design: It's Not What You Said, It's How You Said It
Elaheh Sanoubari, Atil Iscen, Leila Takayama +3
In this paper, we investigate the use of 'prosody' (the musical elements of speech) as a communicative signal for intuitive human-robot interaction interfaces. Our approach, rooted…
Robotic Table Tennis: A Case Study into a High Speed Learning System
David B. D'Ambrosio, Jonathan Abelian, Saminda Abeyruwan +32
We present a deep-dive into a real-world robotic learning system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the abilit…
Barkour: Benchmarking Animal-level Agility with Quadruped Robots
Ken Caluwaerts, Atil Iscen, J. Chase Kew +41
Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biol…
Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions
Alejandro Escontrela, Xue Bin Peng, Wenhao Yu +4
Training a high-dimensional simulated agent with an under-specified reward function often leads the agent to learn physically infeasible strategies that are ineffective when deploy…
Disentangled Planning and Control in Vision Based Robotics via Reward Machines
Alberto Camacho, Jacob Varley, Deepali Jain +2
In this work we augment a Deep Q-Learning agent with a Reward Machine (DQRM) to increase speed of learning vision-based policies for robot tasks, and overcome some of the limitatio…
From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
Deepali Jain, Atil Iscen, Ken Caluwaerts
Legged robots navigating crowded scenes and complex terrains in the real world are required to execute dynamic leg movements while processing visual input for obstacle avoidance an…
Learning Agile Locomotion Skills with a Mentor
Atil Iscen, George Yu, Alejandro Escontrela +3
Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requ…
Zero-Shot Terrain Generalization for Visual Locomotion Policies
Alejandro Escontrela, George Yu, Peng Xu +2
Legged robots have unparalleled mobility on unstructured terrains. However, it remains an open challenge to design locomotion controllers that can operate in a large variety of env…
Policies Modulating Trajectory Generators
Atil Iscen, Ken Caluwaerts, Jie Tan +4
We propose an architecture for learning complex controllable behaviors by having simple Policies Modulate Trajectory Generators (PMTG), a powerful combination that can provide both…
Data Efficient Reinforcement Learning for Legged Robots
Yuxiang Yang, Ken Caluwaerts, Atil Iscen +3
We present a model-based framework for robot locomotion that achieves walking based on only 4.5 minutes (45,000 control steps) of data collected on a quadruped robot. To accurately…
Hierarchical Reinforcement Learning for Quadruped Locomotion
Deepali Jain, Atil Iscen, Ken Caluwaerts
Legged locomotion is a challenging task for learning algorithms, especially when the task requires a diverse set of primitive behaviors. To solve these problems, we introduce a hie…
NoRML: No-Reward Meta Learning
Yuxiang Yang, Ken Caluwaerts, Atil Iscen +2
Efficiently adapting to new environments and changes in dynamics is critical for agents to successfully operate in the real world. Reinforcement learning (RL) based approaches typi…
Provably Robust Blackbox Optimization for Reinforcement Learning
Krzysztof Choromanski, Aldo Pacchiano, Jack Parker-Holder +6
Interest in derivative-free optimization (DFO) and "evolutionary strategies" (ES) has recently surged in the Reinforcement Learning (RL) community, with growing evidence that they…
Optimizing Simulations with Noise-Tolerant Structured Exploration
Krzysztof Choromanski, Atil Iscen, Vikas Sindhwani +2
We propose a simple drop-in noise-tolerant replacement for the standard finite difference procedure used ubiquitously in blackbox optimization. In our approach, parameter perturbat…
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Jie Tan, Tingnan Zhang, Erwin Coumans +5
Designing agile locomotion for quadruped robots often requires extensive expertise and tedious manual tuning. In this paper, we present a system to automate this process by leverag…