Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias

Abstract

Data-driven approaches to solving robotic tasks have gained a lot of traction in recent years. However, most existing policies are trained on large-scale datasets collected in curated lab settings. If we aim to deploy these models in unstructured visual environments like people's homes, they will be unable to cope with the mismatch in data distribution. In such light, we present the first systematic effort in collecting a large dataset for robotic grasping in homes. First, to scale and parallelize data collection, we built a low cost mobile manipulator assembled for under 3K USD. Second, data collected using low cost robots suffer from noisy labels due to imperfect execution and calibration errors. To handle this, we develop a framework which factors out the noise as a latent variable. Our model is trained on 28K grasps collected in several houses under an array of different environmental conditions. We evaluate our models by physically executing grasps on a collection of novel objects in multiple unseen homes. The models trained with our home dataset showed a marked improvement of 43.7% over a baseline model trained with data collected in lab. Our architecture which explicitly models the latent noise in the dataset also performed 10% better than one that did not factor out the noise. We hope this effort inspires the robotics community to look outside the lab and embrace learning based approaches to handle inaccurate cheap robots.

Cited by in corpus (30)

• On the Opportunities and Risks of Foundation Models
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• RUBi: Reducing Unimodal Biases in Visual Question Answering
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
• PyRobot: An Open-source Robotics Framework for Research and Benchmarking
• AllenAct: A Framework for Embodied AI Research
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
• COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning
• Visual Imitation Made Easy
• REPLAB: A Reproducible Low-Cost Arm Benchmark Platform for Robotic Learning
• Learning to Manipulate Deformable Objects without Demonstrations
• Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
• Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping
• RoboNet: Large-Scale Multi-Robot Learning
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
• SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo
• Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
• Can Synthetic Faces Undo the Damage of Dataset Bias to Face Recognition and Facial Landmark Detection?
• Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State Distributions
• Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations
• TAdam: A Robust Stochastic Gradient Optimizer
• Towards Robotic Assembly by Predicting Robust, Precise and Task-oriented Grasps
• EnD: Entangling and Disentangling deep representations for bias correction
• A non-discriminatory approach to ethical deep learning
• Grasp Proposal Networks: An End-to-End Solution for Visual Learning of Robotic Grasps
• A Simple Approach to Continual Learning by Transferring Skill Parameters
• Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint Cables
• Tool as Embodiment for Recursive Manipulation