Reinforcement Learning Algorithms: An Overview and Classification

Abstract

The desire to make applications and machines more intelligent and the aspiration to enable their operation without human interaction have been driving innovations in neural networks, deep learning, and other machine learning techniques. Although reinforcement learning has been primarily used in video games, recent advancements and the development of diverse and powerful reinforcement algorithms have enabled the reinforcement learning community to move from playing video games to solving complex real-life problems in autonomous systems such as self-driving cars, delivery drones, and automated robotics. Understanding the environment of an application and the algorithms' limitations plays a vital role in selecting the appropriate reinforcement learning algorithm that successfully solves the problem on hand in an efficient manner. Consequently, in this study, we identify three main environment types and classify reinforcement learning algorithms according to those environment types. Moreover, within each category, we identify relationships between algorithms. The overview of each algorithm provides insight into the algorithms' foundations and reviews similarities and differences among algorithms. This study provides a perspective on the field and helps practitioners and researchers to select the appropriate algorithm for their use case.

References in corpus (14)

• Continuous control with deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Addressing Function Approximation Error in Actor-Critic Methods
• Prioritized Experience Replay
• Dueling Network Architectures for Deep Reinforcement Learning
• Deep Recurrent Q-Learning for Partially Observable MDPs
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Distributed Distributional Deterministic Policy Gradients
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Sample Efficient Actor-Critic with Experience Replay
• Phasic Policy Gradient

Cited by in corpus (2)

• Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Review
• Reinforcement Learning Pair Trading: A Dynamic Scaling approach