Transfer Learning in Deep Reinforcement Learning: A Survey

Abstract

Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. Along with the promising prospects of reinforcement learning in numerous domains such as robotics and game-playing, transfer learning has arisen to tackle various challenges faced by reinforcement learning, by transferring knowledge from external expertise to facilitate the efficiency and effectiveness of the learning process. In this survey, we systematically investigate the recent progress of transfer learning approaches in the context of deep reinforcement learning. Specifically, we provide a framework for categorizing the state-of-the-art transfer learning approaches, under which we analyze their goals, methodologies, compatible reinforcement learning backbones, and practical applications. We also draw connections between transfer learning and other relevant topics from the reinforcement learning perspective and explore their potential challenges that await future research progress.

References in corpus (40)

• Distilling the Knowledge in a Neural Network
• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Playing Atari with Deep Reinforcement Learning
• Neural Architecture Search with Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Addressing Function Approximation Error in Actor-Critic Methods
• Recent Trends in Deep Learning Based Natural Language Processing
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Deep Q-learning from Demonstrations
• Model compression via distillation and quantization
• An Actor-Critic Algorithm for Sequence Prediction
• Modular Multitask Reinforcement Learning with Policy Sketches
• Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
• Equivalence Between Policy Gradients and Soft Q-Learning
• Control of Memory, Active Perception, and Action in Minecraft
• EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
• Reinforcement Learning in Healthcare: A Survey
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• Reinforcement Learning from Imperfect Demonstrations
• Deep Successor Reinforcement Learning
• Preparing for the Unknown: Learning a Universal Policy with Online System Identification
• Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning
• Kickstarting Deep Reinforcement Learning
• Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
• Multi-Hop Knowledge Graph Reasoning with Reward Shaping
• Decoupling Dynamics and Reward for Transfer Learning
• Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations
• Hierarchical Decision Making In Electricity Grid Management
• Online Transfer Learning in Reinforcement Learning Domains
• Universal Successor Features Approximators
• Reward Shaping with Recurrent Neural Networks for Speeding up On-Line Policy Learning in Spoken Dialogue Systems
• State Alignment-based Imitation Learning
• Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations
• Disentangled Skill Embeddings for Reinforcement Learning
• Off-Policy Imitation Learning from Observations

Cited by in corpus (25)

• A Review of Deep Reinforcement Learning for Smart Building Energy Management
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directions
• Transfer Learning for Future Wireless Networks: A Comprehensive Survey
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning
• Network Slicing via Transfer Learning aided Distributed Deep Reinforcement Learning
• AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning
• Toward Efficient Transfer Learning in 6G
• Reinforcement Learning by Guided Safe Exploration
• Inter-Cell Network Slicing With Transfer Learning Empowered Multi-Agent Deep Reinforcement Learning
• Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants
• Distributed Deep Learning in Open Collaborations
• Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search
• Block Contextual MDPs for Continual Learning
• Fractional Transfer Learning for Deep Model-Based Reinforcement Learning
• The Effect of Q-function Reuse on the Total Regret of Tabular, Model-Free, Reinforcement Learning
• Transferring Reinforcement Learning for DC-DC Buck Converter Control via Duty Ratio Mapping: From Simulation to Implementation
• Offline RL With Resource Constrained Online Deployment
• Provably Efficient Multi-Task Reinforcement Learning with Model Transfer
• Pattern Transfer Learning for Reinforcement Learning in Order Dispatching
• Successor Feature Neural Episodic Control
• Collaboration Promotes Group Resilience in Multi-Agent RL
• Component Transfer Learning for Deep RL Based on Abstract Representations
• Can Q-Learning be Improved with Advice?