A View on Deep Reinforcement Learning in System Optimization

Abstract

Many real-world systems problems require reasoning about the long term consequences of actions taken to configure and manage the system. These problems with delayed and often sequentially aggregated reward, are often inherently reinforcement learning problems and present the opportunity to leverage the recent substantial advances in deep reinforcement learning. However, in some cases, it is not clear why deep reinforcement learning is a good fit for the problem. Sometimes, it does not perform better than the state-of-the-art solutions. And in other cases, random search or greedy algorithms could outperform deep reinforcement learning. In this paper, we review, discuss, and evaluate the recent trends of using deep reinforcement learning in system optimization. We propose a set of essential metrics to guide future works in evaluating the efficacy of using deep reinforcement learning in system optimization. Our evaluation includes challenges, the types of problems, their formulation in the deep reinforcement learning setting, embedding, the model used, efficiency, and robustness. We conclude with a discussion on open challenges and potential directions for pushing further the integration of reinforcement learning in system optimization.

References in corpus (15)

• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• Neo: A Learned Query Optimizer
• Deep Q-learning from Demonstrations
• Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-time Popularities
• A Survey on Compiler Autotuning using Machine Learning
• Dopamine: A Research Framework for Deep Reinforcement Learning
• Reinforcement Learning: A Survey
• Learning to Optimize Join Queries With Deep Reinforcement Learning
• Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
• Reinforced Genetic Algorithm Learning for Optimizing Computation Graphs
• Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning
• Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control

Cited by in corpus (3)

• Two-stage WECC Composite Load Modeling: A Double Deep Q-Learning Networks Approach
• AutoPhase: Juggling HLS Phase Orderings in Random Forests with Deep Reinforcement Learning
• Byzantine-Fault-Tolerant Consensus via Reinforcement Learning for Permissioned Blockchain Implemented in a V2X Network