A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms

Abstract

A key challenge in intelligent robotics is creating robots that are capable of directly interacting with the world around them to achieve their goals. The last decade has seen substantial growth in research on the problem of robot manipulation, which aims to exploit the increasing availability of affordable robot arms and grippers to create robots capable of directly interacting with the world to achieve their goals. Learning will be central to such autonomous systems, as the real world contains too much variation for a robot to expect to have an accurate model of its environment, the objects in it, or the skills required to manipulate them, in advance. We aim to survey a representative subset of that research which uses machine learning for manipulation. We describe a formalization of the robot manipulation learning problem that synthesizes existing research into a single coherent framework and highlight the many remaining research opportunities and challenges.

References in corpus (54)

• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Gaussian Processes for Data-Efficient Learning in Robotics and Control
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• On First-Order Meta-Learning Algorithms
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Cooperative Inverse Reinforcement Learning
• More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
• Hindsight Experience Replay
• Continuous Deep Q-Learning with Model-based Acceleration
• Maximum Entropy Deep Inverse Reinforcement Learning
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• Learning by Playing - Solving Sparse Reward Tasks from Scratch
• FeUdal Networks for Hierarchical Reinforcement Learning
• Data-Efficient Hierarchical Reinforcement Learning
• Interactive Policy Learning through Confidence-Based Autonomy
• Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control
• Go-Explore: a New Approach for Hard-Exploration Problems
• Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
• RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter
• Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
• Temporal Difference Models: Model-Free Deep RL for Model-Based Control
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• Third-Person Imitation Learning
• FFRob: Leveraging Symbolic Planning for Efficient Task and Motion Planning
• Evolved Policy Gradients
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
• Grasp2Vec: Learning Object Representations from Self-Supervised Grasping
• Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning
• Learning Plannable Representations with Causal InfoGAN
• Learning Multi-Level Hierarchies with Hindsight
• Safe Policy Improvement by Minimizing Robust Baseline Regret
• Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints
• Environment-Independent Task Specifications via GLTL
• One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks
• Riemannian Motion Policies
• Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
• Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
• End-to-End Learning of Semantic Grasping
• Few-Shot Goal Inference for Visuomotor Learning and Planning
• Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
• Learning Robotic Manipulation of Granular Media
• Efficient Hierarchical Robot Motion Planning Under Uncertainty and Hybrid Dynamics
• Hybrid Bayesian Eigenobjects: Combining Linear Subspace and Deep Network Methods for 3D Robot Vision
• Unsupervised Learning for Nonlinear PieceWise Smooth Hybrid Systems
• Safety Augmented Value Estimation from Demonstrations (SAVED): Safe Deep Model-Based RL for Sparse Cost Robotic Tasks
• Hybrid DDP in Clutter (CHDDP): Trajectory Optimization for Hybrid Dynamical System in Cluttered Environments

Cited by in corpus (20)

• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• Learning Force Control for Contact-rich Manipulation Tasks with Rigid Position-controlled Robots
• Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning
• Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks
• Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation
• Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations
• A Survey of Embodied Learning for Object-Centric Robotic Manipulation
• A Survey on the Integration of Machine Learning with Sampling-based Motion Planning
• Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics
• Experience-Embedded Visual Foresight
• PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards
• Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulation
• Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materials
• Learning a Generative Transition Model for Uncertainty-Aware Robotic Manipulation
• Pixyz: a Python library for developing deep generative models
• Learning to Play by Imitating Humans
• Dataset Bias Mitigation Through Analysis of CNN Training Scores
• Automatic Failure Recovery for End-User Programs on Service Mobile Robots
• CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context
• An Adaptive Framework for Reliable Trajectory Following in Changing-Contact Robot Manipulation Tasks