Transfer Learning in Deep Reinforcement Learning: A Survey

Abstract

Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. Along with the promising prospects of reinforcement learning in numerous domains such as robotics and game-playing, transfer learning has arisen to tackle various challenges faced by reinforcement learning, by transferring knowledge from external expertise to facilitate the efficiency and effectiveness of the learning process. In this survey, we systematically investigate the recent progress of transfer learning approaches in the context of deep reinforcement learning. Specifically, we provide a framework for categorizing the state-of-the-art transfer learning approaches, under which we analyze their goals, methodologies, compatible reinforcement learning backbones, and practical applications. We also draw connections between transfer learning and other relevant topics from the reinforcement learning perspective and explore their potential challenges that await future research progress.

References in corpus (15)

• Distilling the Knowledge in a Neural Network
• Neural Architecture Search with Reinforcement Learning
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• An Actor-Critic Algorithm for Sequence Prediction
• EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning
• Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
• Online Transfer Learning in Reinforcement Learning Domains
• Universal Successor Features Approximators
• State Alignment-based Imitation Learning
• Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations
• Disentangled Skill Embeddings for Reinforcement Learning
• Off-Policy Imitation Learning from Observations

Cited by in corpus (25)

• A Review of Deep Reinforcement Learning for Smart Building Energy Management
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directions
• Transfer Learning for Future Wireless Networks: A Comprehensive Survey
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning
• Network Slicing via Transfer Learning aided Distributed Deep Reinforcement Learning
• AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning
• Toward Efficient Transfer Learning in 6G
• Reinforcement Learning by Guided Safe Exploration
• Inter-Cell Network Slicing With Transfer Learning Empowered Multi-Agent Deep Reinforcement Learning
• Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants
• Distributed Deep Learning in Open Collaborations
• Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search
• Block Contextual MDPs for Continual Learning
• Fractional Transfer Learning for Deep Model-Based Reinforcement Learning
• The Effect of Q-function Reuse on the Total Regret of Tabular, Model-Free, Reinforcement Learning
• Transferring Reinforcement Learning for DC-DC Buck Converter Control via Duty Ratio Mapping: From Simulation to Implementation
• Offline RL With Resource Constrained Online Deployment
• Provably Efficient Multi-Task Reinforcement Learning with Model Transfer
• Pattern Transfer Learning for Reinforcement Learning in Order Dispatching
• Successor Feature Neural Episodic Control
• Collaboration Promotes Group Resilience in Multi-Agent RL
• Component Transfer Learning for Deep RL Based on Abstract Representations
• Can Q-Learning be Improved with Advice?