Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

Abstract

In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in isolation such as grasping, pushing or poking. In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc. But how does one collect such a dataset? In this paper, we present the largest available robotic-demonstration dataset (MIME) that contains 8260 human-robot demonstrations over 20 different robotic tasks (https://sites.google.com/view/mimedataset). These tasks range from the simple task of pushing objects to the difficult task of stacking household objects. Our dataset consists of videos of human demonstrations and kinesthetic trajectories of robot demonstrations. We also propose to use this dataset for the task of mapping 3rd person video features to robot trajectories. Furthermore, we present two different approaches using this dataset and evaluate the predicted robot trajectories against ground-truth trajectories. We hope our dataset inspires research in multiple areas including visual imitation, trajectory prediction, and multi-task robotic learning.

arXiv comment10 pages, CoRL 2018

Cited by in corpus (16)

• On the Opportunities and Risks of Foundation Models
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
• Scaling data-driven robotics with reward sketching and batch reinforcement learning
• Visual Imitation Made Easy
• RoboNet: Large-Scale Multi-Robot Learning
• Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
• Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity
• Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations
• O2A: One-shot Observational learning with Action vectors
• Reinforcement Learning with Videos: Combining Offline Observations with Interaction
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
• Learning Feasibility to Imitate Demonstrators with Different Dynamics
• Conservative Data Sharing for Multi-Task Offline Reinforcement Learning
• Shaping embodied agent behavior with activity-context priors from egocentric video
• Single RGB-D Camera Teleoperation for General Robotic Manipulation