Deep Reinforcement Learning: An Overview
arXiv:1701.07274
Abstract
We give an overview of recent exciting achievements of deep reinforcement learning (RL). We discuss six core elements, six important mechanisms, and twelve applications. We start with background of machine learning, deep learning and reinforcement learning. Next we discuss core RL elements, including value function, in particular, Deep Q-Network (DQN), policy, reward, model, planning, and exploration. After that, we discuss important mechanisms for RL, including attention and memory, unsupervised learning, transfer learning, multi-agent RL, hierarchical RL, and learning to learn. Then we discuss various applications of RL, including games, in particular, AlphaGo, robotics, natural language processing, including dialogue systems, machine translation, and text generation, computer vision, neural architecture design, business management, finance, healthcare, Industry 4.0, smart grid, intelligent transportation systems, and computer systems. We mention topics not reviewed yet, and list a collection of RL resources. After presenting a brief summary, we close with discussions. Please see Deep Reinforcement Learning, arXiv:1810.06339, for a significant update.
arXiv commentPlease see Deep Reinforcement Learning, arXiv:1810.06339, for a significant update
References in corpus (137)
- Efficient Estimation of Word Representations in Vector Space
- Deep Learning in Neural Networks: An Overview
- Auto-Encoding Variational Bayes
- Sequence to Sequence Learning with Neural Networks
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Continuous control with deep reinforcement learning
- Overcoming catastrophic forgetting in neural networks
- Deep Learning with Differential Privacy
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Neural Architecture Search with Reinforcement Learning
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Towards A Rigorous Science of Interpretable Machine Learning
- Learning Transferable Features with Deep Adaptation Networks
- A Contextual-Bandit Approach to Personalized News Article Recommendation
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Dueling Network Architectures for Deep Reinforcement Learning
- Improved Training of Wasserstein GANs
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- Unsupervised Domain Adaptation with Residual Transfer Networks
- Learning Combinatorial Optimization Algorithms over Graphs
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
- Recurrent Models of Visual Attention
- DRAW: A Recurrent Neural Network For Image Generation
- Benchmarking Deep Reinforcement Learning for Continuous Control
- Matching Networks for One Shot Learning
- Visualizing and Understanding Recurrent Networks
- Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
- Deep Convolutional Inverse Graphics Network
- Multiple Object Recognition with Visual Attention
- Dual Learning for Machine Translation
- Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- Dynamic Coattention Networks For Question Answering
- Deep reinforcement learning from human preferences
- VIME: Variational Information Maximizing Exploration
- Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
- Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
- Deep Exploration via Bootstrapped DQN
- Action-Conditional Video Prediction using Deep Networks in Atari Games
- Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
- Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
- Deep Reinforcement Learning for Dialogue Generation
- Massively Parallel Methods for Deep Reinforcement Learning
- Cooperative Inverse Reinforcement Learning
- Learning to reinforcement learn
- Learning to Navigate in Complex Environments
- Generative Adversarial Imitation Learning
- The Microsoft 2016 Conversational Speech Recognition System
- Hindsight Experience Replay
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
- #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
- Continuous Deep Q-Learning with Model-based Acceleration
- Deep Q-learning from Demonstrations
- Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
- ReasoNet: Learning to Stop Reading in Machine Comprehension
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
- Multi-agent Reinforcement Learning in Sequential Social Dilemmas
- Learning Multiagent Communication with Backpropagation
- Unifying Count-Based Exploration and Intrinsic Motivation
- Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
- Learning to Remember Rare Events
- DeepCoder: Learning to Write Programs
- Neural Programmer-Interpreters
- Device Placement Optimization with Reinforcement Learning
- An Actor-Critic Algorithm for Sequence Prediction
- Bridging the Gap Between Value and Policy Based Reinforcement Learning
- Modular Multitask Reinforcement Learning with Policy Sketches
- Imagination-Augmented Agents for Deep Reinforcement Learning
- Neural Optimizer Search with Reinforcement Learning
- Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control
- Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
- Hybrid Reward Architecture for Reinforcement Learning
- Learning Representations for Counterfactual Inference
- Learning Multiple Tasks with Multilinear Relationship Networks
- Stochastic Neural Networks for Hierarchical Reinforcement Learning
- Distral: Robust Multitask Reinforcement Learning
- A Berkeley View of Systems Challenges for AI
- Control of Memory, Active Perception, and Action in Minecraft
- Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning
- Using Fast Weights to Attend to the Recent Past
- Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
- The Predictron: End-To-End Learning and Planning
- Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
- Third-Person Imitation Learning
- A Deep Hierarchical Approach to Lifelong Learning in Minecraft
- InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
- Learning to Poke by Poking: Experiential Learning of Intuitive Physics
- Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning
- Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
- Federated Multi-Task Learning
- DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning
- Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
- Associative Long Short-Term Memory
- Language Understanding for Text-based Games Using Deep Reinforcement Learning
- Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks
- Making Neural Programming Architectures Generalize via Recursion
- Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
- Learning values across many orders of magnitude
- Tree-Structured Reinforcement Learning for Sequential Object Localization
- Deep Probabilistic Programming
- On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems
- Strategic Attentive Writer for Learning Macro-Actions
- Learning through Dialogue Interactions by Asking Questions
- A multi-agent reinforcement learning model of common-pool resource appropriation
- Failures of Gradient-Based Deep Learning
- Deep Reinforcement Learning-based Image Captioning with Embedding Reward
- Inverse Reward Design
- Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and Control
- Learning to Perform Physics Experiments via Deep Reinforcement Learning
- A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning
- Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
- A Benchmark Environment Motivated by Industrial Control Problems
- Learning how to Active Learn: A Deep Reinforcement Learning Approach
- Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening
- Dialogue Learning With Human-In-The-Loop
- Learning to Compose Words into Sentences with Reinforcement Learning
- Attention-Aware Face Hallucination via Deep Reinforcement Learning
- Value Prediction Network
- Minimax Regret Bounds for Reinforcement Learning
- Model-Free Imitation Learning with Policy Optimization
- From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood
- Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
- Generalizing Skills with Semi-Supervised Reinforcement Learning
- Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learning
- STARDATA: A StarCraft AI Research Dataset
- Repeated Inverse Reinforcement Learning
- Differentially Private Policy Evaluation
- Deep Visual Foresight for Planning Robot Motion
- 3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
- Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning
- Improving Interpretability of Deep Neural Networks with Semantic Information
- Teaching a Machine to Read Maps with Deep Reinforcement Learning
- Investigating practical linear temporal difference learning
- Predictive State Recurrent Neural Networks
- Predictive-State Decoders: Encoding the Future into Recurrent Networks
Cited by in corpus (134)
- A Brief Survey of Deep Reinforcement Learning
- Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
- Applications of Deep Learning and Reinforcement Learning to Biological Data
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- A Review of Deep Reinforcement Learning for Smart Building Energy Management
- Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving
- The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
- Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
- Medical Deep Learning -- A systematic Meta-Review
- A Survey of Deep RL and IL for Autonomous Driving Policy Learning
- An Application of Deep Reinforcement Learning to Algorithmic Trading
- Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
- A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization
- Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review
- A Survey of Deep Reinforcement Learning in Video Games
- Game-Theoretic Multiagent Reinforcement Learning
- Deep Learning in Mobile and Wireless Networking: A Survey
- Levels of explainable artificial intelligence for human-aligned conversational explanations
- Recent Advances in Deep Learning: An Overview
- Review: Deep Learning in Electron Microscopy
- Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis
- Deep Reinforcement Learning for Black-Box Testing of Android Apps
- Evolutionary Reinforcement Learning: A Survey
- Neural Policy Gradient Methods: Global Optimality and Rates of Convergence
- Ensemble-Based Deep Reinforcement Learning for Chatbots
- Interplay between RIS and AI in Wireless Communications: Fundamentals, Architectures, Applications, and Open Research Problems
- Quantum simulation with hybrid tensor networks
- Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
- DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
- Adaptive Behavior Generation for Autonomous Driving using Deep Reinforcement Learning with Compact Semantic States
- Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
- Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
- Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues
- Reinforcement Learning Applications
- From the digital data revolution to digital health and digital economy toward a digital society: Pervasiveness of Artificial Intelligence
- A Survey of Optimization Methods from a Machine Learning Perspective
- PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning
- Sample Efficient Policy Gradient Methods with Recursive Variance Reduction
- LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations
- Recurrent Deterministic Policy Gradient Method for Bipedal Locomotion on Rough Terrain Challenge
- RLCard: A Toolkit for Reinforcement Learning in Card Games
- Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning
- RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning
- A Deep Q-Network for the Beer Game: A Deep Reinforcement Learning algorithm to Solve Inventory Optimization Problems
- Stability-certified reinforcement learning: A control-theoretic perspective
- Deep Reinforcement Learning for Human-Like Driving Policies in Collision Avoidance Tasks of Self-Driving Cars
- Machine Learning in Compiler Optimisation
- Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
- Deep Reinforcement Learning and Transportation Research: A Comprehensive Review
- Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning
- Rafiki: Machine Learning as an Analytics Service System
- Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy
- Deep Reinforcement One-Shot Learning for Artificially Intelligent Classification Systems
- Low-Rank Bottleneck in Multi-head Attention Models
- Towards Symbolic Reinforcement Learning with Common Sense
- Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches
- Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey
- DRIFT: Deep Reinforcement Learning for Functional Software Testing
- Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
- Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight
- Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
- Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Games
- Map-based Multi-Policy Reinforcement Learning: Enhancing Adaptability of Robots by Deep Reinforcement Learning
- Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks
- Causality and Batch Reinforcement Learning: Complementary Approaches To Planning In Unknown Domains
- Attention Control with Metric Learning Alignment for Image Set-based Recognition
- Dependency-aware Attention Control for Unconstrained Face Recognition with Image Sets
- An Intelligent Control Strategy for buck DC-DC Converter via Deep Reinforcement Learning
- A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization
- Multi-Agent Deep Reinforcement Learning Based Trajectory Planning for Multi-UAV Assisted Mobile Edge Computing
- Towards Scalable Verification of Deep Reinforcement Learning
- Active Feature Acquisition with Generative Surrogate Models
- Generosity, selfishness and exploitation as optimal greedy strategies for resource sharing
- Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving
- Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning
- Using Monte Carlo Tree Search as a Demonstrator within Asynchronous Deep RL
- Deep Reinforcement Learning for Active High Frequency Trading
- Continuous Doubly Constrained Batch Reinforcement Learning
- RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement Pathway Optimization
- Learning Index Selection with Structured Action Spaces
- Developing an OpenAI Gym-compatible framework and simulation environment for testing Deep Reinforcement Learning agents solving the Ambulance Location Problem
- Safer Deep RL with Shallow MCTS: A Case Study in Pommerman
- A Deep Reinforcement Learning Approach to Efficient Drone Mobility Support
- AI based Algorithms of Path Planning, Navigation and Control for Mobile Ground Robots and UAVs
- Multi-Point Bandit Algorithms for Nonstationary Online Nonconvex Optimization
- Hierarchical principles of embodied reinforcement learning: A review
- Models and Framework for Adversarial Attacks on Complex Adaptive Systems
- Crowdsensing Game with Demand Uncertainties: A Deep Reinforcement Learning Approach
- What Would You Do? Acting by Learning to Predict
- Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound
- Towards Optimal District Heating Temperature Control in China with Deep Reinforcement Learning
- Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research
- Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging
- Distilling Information from a Flood: A Possibility for the Use of Meta-Analysis and Systematic Review in Machine Learning Research
- Learning Agents With Prioritization and Parameter Noise in Continuous State and Action Space
- Avoidance of Manual Labeling in Robotic Autonomous Navigation Through Multi-Sensory Semi-Supervised Learning
- Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey
- Reducing catastrophic forgetting when evolving neural networks
- A Novel Deep Neural Network Architecture for Mars Visual Navigation
- Spark-MPI: Approaching the Fifth Paradigm of Cognitive Applications
- Using Distance Correlation for Efficient Bayesian Optimization
- Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning
- Carrier-Sense Multiple Access for Heterogeneous Wireless Networks Using Deep Reinforcement Learning
- AlphaSeq: Sequence Discovery with Deep Reinforcement Learning
- Concept and the implementation of a tool to convert industry 4.0 environments modeled as FSM to an OpenAI Gym wrapper
- Deep Reinforcement Learning with Linear Quadratic Regulator Regions
- Why? Why not? When? Visual Explanations of Agent Behavior in Reinforcement Learning
- Bregman Gradient Policy Optimization
- USCO-Solver: Solving Undetermined Stochastic Combinatorial Optimization Problems
- Learn Task First or Learn Human Partner First: A Hierarchical Task Decomposition Method for Human-Robot Cooperation
- Controlling an Inverted Pendulum with Policy Gradient Methods-A Tutorial
- Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks
- Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets
- Deep Reinforcement Learning for Personalized Search Story Recommendation
- Prioritized Guidance for Efficient Multi-Agent Reinforcement Learning Exploration
- Scaffolding Reflection in Reinforcement Learning Framework for Confinement Escape Problem
- Image Synthesis for Data Augmentation in Medical CT using Deep Reinforcement Learning
- Deep Q learning for fooling neural networks
- Tackling Variabilities in Autonomous Driving
- Optimizing Sponsored Search Ranking Strategy by Deep Reinforcement Learning
- Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks
- Service Chain Composition with Failures in NFV Systems: A Game-Theoretic Perspective
- Autonomous Industrial Management via Reinforcement Learning: Self-Learning Agents for Decision-Making -- A Review
- Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments
- Mix and Match: Markov Chains & Mixing Times for Matching in Rideshare
- When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
- BF++: a language for general-purpose program synthesis
- AME Blockchain: An Architecture Design for Closed-Loop Fluid Economy Token System
- Riemannian Proximal Policy Optimization
- Experience enrichment based task independent reward model
- Online Virtual Machine Allocation with Predictions
- Deep reinforcement learning for RAN optimization and control
- Adaptation of Quadruped Robot Locomotion with Meta-Learning
- PipeTune: Pipeline Parallelism of Hyper and System Parameters Tuning for Deep Learning Clusters