Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening

Abstract

We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the training time. We evaluate the performance of our approach on the 49 games of the challenging Arcade Learning Environment, and report significant improvements in both training time and accuracy.

References in corpus (4)

• Playing Atari with Deep Reinforcement Learning
• Deep Exploration via Bootstrapped DQN
• Massively Parallel Methods for Deep Reinforcement Learning
• Model-Free Episodic Control

Cited by in corpus (24)

• Addressing Function Approximation Error in Actor-Critic Methods
• Deep Reinforcement Learning: An Overview
• Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
• A Survey of Deep Reinforcement Learning in Video Games
• Neural Episodic Control
• Constrained Deep Networks: Lagrangian Optimization via Log-Barrier Extensions
• Learning What Data to Learn
• Constrained domain adaptation for Image segmentation
• Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
• A Data-Efficient Deep Learning Approach for Deployable Multimodal Social Robots
• Generalizable Episodic Memory for Deep Reinforcement Learning
• Analysing Results from AI Benchmarks: Key Indicators and How to Obtain Them
• Sample-Efficient Deep Reinforcement Learning via Episodic Backward Update
• Regularized Softmax Deep Multi-Agent $Q$-Learning
• Estimation Error Correction in Deep Reinforcement Learning for Deterministic Actor-Critic Methods
• Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning
• Self-Imitation Learning via Generalized Lower Bound Q-learning
• Self-Imitation Advantage Learning
• On the Reduction of Variance and Overestimation of Deep Q-Learning
• An Inductive Bias for Distances: Neural Nets that Respect the Triangle Inequality
• Episodic Memory Deep Q-Networks
• Non-parametric Uni-modality Constraints for Deep Ordinal Classification
• Deep Echo State Q-Network (DEQN) and Its Application in Dynamic Spectrum Sharing for 5G and Beyond
• Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policy