Review, Analysis and Design of a Comprehensive Deep Reinforcement Learning Framework

Abstract

The integration of deep learning to reinforcement learning (RL) has enabled RL to perform efficiently in high-dimensional environments. Deep RL methods have been applied to solve many complex real-world problems in recent years. However, development of a deep RL-based system is challenging because of various issues such as the selection of a suitable deep RL algorithm, its network configuration, training time, training methods, and so on. This paper proposes a comprehensive software framework that not only plays a vital role in designing a connect-the-dots deep RL architecture but also provides a guideline to develop a realistic RL application in a short time span. We have designed and developed a deep RL-based software framework that strictly ensures flexibility, robustness, and scalability. By inheriting the proposed architecture, software managers can foresee any challenges when designing a deep RL-based system. As a result, they can expedite the design process and actively control every stage of software development, which is especially critical in agile development environments. To enforce generalization, the proposed architecture does not depend on a specific RL algorithm, a network configuration, the number of agents, or the type of agents. Using our framework, software developers can develop and integrate new RL algorithms or new types of agents, and can flexibly change network configuration or the number of agents.

References in corpus (41)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Continuous control with deep reinforcement learning
• Overcoming catastrophic forgetting in neural networks
• Playing Atari with Deep Reinforcement Learning
• Trust Region Policy Optimization
• Prioritized Experience Replay
• Dueling Network Architectures for Deep Reinforcement Learning
• Rényi Divergence and Kullback-Leibler Divergence
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Theoretical Models of Learning to Learn
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Deep Reinforcement Learning for Cyber Security
• Distributed Prioritized Experience Replay
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Large-Scale Study of Curiosity-Driven Learning
• Count-Based Exploration with Neural Density Models
• Hindsight Experience Replay
• Bridging the Gap Between Value and Policy Based Reinforcement Learning
• On-line Policy Improvement using Monte-Carlo Search
• Sample Efficient Actor-Critic with Experience Replay
• Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
• Hybrid Reward Architecture for Reinforcement Learning
• Equivalence Between Policy Gradients and Soft Q-Learning
• Dopamine: A Research Framework for Deep Reinforcement Learning
• Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
• A Multi-Objective Deep Reinforcement Learning Framework
• Bayes' Bluff: Opponent Modelling in Poker
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Deep Attention Recurrent Q-Network
• Multi-Objective Deep Reinforcement Learning
• Accelerated Methods for Deep Reinforcement Learning
• Combining policy gradient and Q-learning
• Mutual Alignment Transfer Learning
• A Human Mixed Strategy Approach to Deep Reinforcement Learning