588 citations · 734 across the 17 of their papers we have counts for
19 papers
Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer
Gemini Robotics Team, Abbas Abdolmaleki, Saminda Abeyruwan +169
General-purpose robots need a deep understanding of the physical world, advanced reasoning, and general and dexterous control. This report introduces the latest generation of the G…
ExoStart: Efficient learning for dexterous manipulation with sensorized exoskeleton demonstrations
Zilin Si, Jose Enrique Chen, M. Emre Karagozler +15
Recent advancements in teleoperation systems have enabled high-quality data collection for robotic manipulators, showing impressive results in learning manipulation at scale. This…
Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie +115
Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as…
DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots
Maria Bauza, Jose Enrique Chen, Valentin Dalibard +13
We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand…
RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao +36
The ability to leverage heterogeneous robotic experience from different robots and tasks to quickly master novel skills and embodiments has the potential to transform robot learnin…
How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
Alex X. Lee, Coline Devin, Jost Tobias Springenberg +4
Reinforcement learning (RL) has been shown to be effective at learning control from experience. However, RL typically requires a large amount of online interaction with the environ…
Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes
Alex X. Lee, Coline Devin, Yuxiang Zhou +18
We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategie…
Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner
Philemon Brakel, Steven Bohez, Leonard Hasenclever +2
Dynamic quadruped locomotion over challenging terrains with precise foot placements is a hard problem for both optimal control methods and Reinforcement Learning (RL). Non-linear s…
On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning
Abbas Abdolmaleki, Sandy H. Huang, Giulia Vezzani +11
Many advances that have improved the robustness and efficiency of deep reinforcement learning (RL) algorithms can, in one way or another, be understood as introducing additional ob…
Manipulator-Independent Representations for Visual Imitation
Yuxiang Zhou, Yusuf Aytar, Konstantinos Bousmalis
Imitation learning is an effective tool for robotic learning tasks where specifying a reinforcement learning (RL) reward is not feasible or where the exploration problem is particu…
Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation
Rae Jeong, Yusuf Aytar, David Khosid +5
Collecting and automatically obtaining reward signals from real robotic visual data for the purposes of training reinforcement learning algorithms can be quite challenging and time…
Off-Policy Evaluation via Off-Policy Classification
Alex Irpan, Kanishka Rao, Konstantinos Bousmalis +3
In this work, we consider the problem of model selection for deep reinforcement learning (RL) in real-world environments. Typically, the performance of deep RL algorithms is evalua…
Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks
Stephen James, Paul Wohlhart, Mrinal Kalakrishnan +6
Real world data, especially in the domain of robotics, is notoriously costly to collect. One way to circumvent this can be to leverage the power of simulation to produce large amou…
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach
Behnam Gholami, Pritish Sahu, Ognjen Rudovic +2
Unsupervised domain adaptation (uDA) models focus on pairwise adaptation settings where there is a single, labeled, source and a single target domain. However, in many real-world s…
XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings
Amélie Royer, Konstantinos Bousmalis, Stephan Gouws +4
Style transfer usually refers to the task of applying color and texture information from a specific style image to a given content image while preserving the structure of the latte…
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
Konstantinos Bousmalis, Alex Irpan, Paul Wohlhart +9
Instrumenting and collecting annotated visual grasping datasets to train modern machine learning algorithms can be extremely time-consuming and expensive. An appealing alternative…
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Konstantinos Bousmalis, Nathan Silberman, David Dohan +2
Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks. One appealing alternative is rendering synthetic dat…
Domain Separation Networks
Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman +2
The cost of large scale data collection and annotation often makes the application of machine learning algorithms to new tasks or datasets prohibitively expensive. One approach cir…
A deep matrix factorization method for learning attribute representations
George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou +1
Semi-Non-negative Matrix Factorization is a technique that learns a low-dimensional representation of a dataset that lends itself to a clustering interpretation. It is possible tha…