A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

Abstract

Deep learning techniques have been widely applied, achieving state-of-the-art results in various fields of study. This survey focuses on deep learning solutions that target learning control policies for robotics applications. We carry out our discussions on the two main paradigms for learning control with deep networks: deep reinforcement learning and imitation learning. For deep reinforcement learning (DRL), we begin from traditional reinforcement learning algorithms, showing how they are extended to the deep context and effective mechanisms that could be added on top of the DRL algorithms. We then introduce representative works that utilize DRL to solve navigation and manipulation tasks in robotics. We continue our discussion on methods addressing the challenge of the reality gap for transferring DRL policies trained in simulation to real-world scenarios, and summarize robotics simulation platforms for conducting DRL research. For imitation leaning, we go through its three main categories, behavior cloning, inverse reinforcement learning and generative adversarial imitation learning, by introducing their formulations and their corresponding robotics applications. Finally, we discuss the open challenges and research frontiers.

19 pages, 1 figures

References in corpus (39)

• Deep Learning in Neural Networks: An Overview
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Deep Domain Confusion: Maximizing for Domain Invariance
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Emergence of Locomotion Behaviours in Rich Environments
• Noisy Networks for Exploration
• Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
• Learning to reinforcement learn
• Hindsight Experience Replay
• AI2-THOR: An Interactive 3D Environment for Visual AI
• CARLA: An Open Urban Driving Simulator
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network
• One-Shot Visual Imitation Learning via Meta-Learning
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Convolutional Neural Network-based Place Recognition
• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
• Virtual to Real Reinforcement Learning for Autonomous Driving
• Third-Person Imitation Learning
• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
• Reverse Curriculum Generation for Reinforcement Learning
• Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
• Neural SLAM: Learning to Explore with External Memory
• Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation
• Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots
• Unifying Map and Landmark Based Representations for Visual Navigation
• One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
• Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
• Socially Aware Motion Planning with Deep Reinforcement Learning
• Memory Augmented Control Networks
• Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
• Deep Learning a Grasp Function for Grasping under Gripper Pose Uncertainty
• Model-based Adversarial Imitation Learning
• 3D Fully Convolutional Network for Vehicle Detection in Point Cloud
• Deep Visual Foresight for Planning Robot Motion
• Deep Reinforcement Learning for Tensegrity Robot Locomotion

Cited by in corpus (10)

• SCALE-Sim: Systolic CNN Accelerator Simulator
• Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation
• Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments
• Learning to Navigate in Indoor Environments: from Memorizing to Reasoning
• COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learning
• Coordinated Heterogeneous Distributed Perception based on Latent Space Representation
• Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning
• Pose Imitation Constraints for Collaborative Robots
• Learning How to Trade-Off Safety with Agility Using Deep Covariance Estimation for Perception Driven UAV Motion Planning
• Learning from Interventions using Hierarchical Policies for Safe Learning