Dopamine: A Research Framework for Deep Reinforcement Learning

Abstract

Deep reinforcement learning (deep RL) research has grown significantly in recent years. A number of software offerings now exist that provide stable, comprehensive implementations for benchmarking. At the same time, recent deep RL research has become more diverse in its goals. In this paper we introduce Dopamine, a new research framework for deep RL that aims to support some of that diversity. Dopamine is open-source, TensorFlow-based, and provides compact and reliable implementations of some state-of-the-art deep RL agents. We complement this offering with a taxonomy of the different research objectives in deep RL research. While by no means exhaustive, our analysis highlights the heterogeneity of research in the field, and the value of frameworks such as ours.

References in corpus (3)

• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Distributed Prioritized Experience Replay
• TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow

Cited by in corpus (78)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Model-Based Reinforcement Learning for Atari
• The Hanabi Challenge: A New Frontier for AI Research
• Soft Actor-Critic for Discrete Action Settings
• Benchmarking Batch Deep Reinforcement Learning Algorithms
• Dream to Control: Learning Behaviors by Latent Imagination
• Tianshou: a Highly Modularized Deep Reinforcement Learning Library
• Review: Deep Learning in Electron Microscopy
• An Optimistic Perspective on Offline Reinforcement Learning
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• ChainerRL: A Deep Reinforcement Learning Library
• DeepMDP: Learning Continuous Latent Space Models for Representation Learning
• Hyperbolic Discounting and Learning over Multiple Horizons
• rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
• RecSim: A Configurable Simulation Platform for Recommender Systems
• AllenAct: A Framework for Embodied AI Research
• d3rlpy: An Offline Deep Reinforcement Learning Library
• Measuring the Reliability of Reinforcement Learning Algorithms
• Deep Reinforcement Learning at the Edge of the Statistical Precipice
• DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction
• Pretraining Representations for Data-Efficient Reinforcement Learning
• Prioritized Sequence Experience Replay
• Behaviour Suite for Reinforcement Learning
• SEED RL: Scalable and Efficient Deep-RL with Accelerated Central Inference
• A Geometric Perspective on Optimal Representations for Reinforcement Learning
• Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodology
• Mastering Atari with Discrete World Models
• Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review
• Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
• Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
• On Bonus-Based Exploration Methods in the Arcade Learning Environment
• Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research
• Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing field
• Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation
• A View on Deep Reinforcement Learning in System Optimization
• CopyCAT: Taking Control of Neural Policies with Constant Attacks
• On the Estimation Bias in Double Q-Learning
• TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning
• Simplifying Deep Reinforcement Learning via Self-Supervision
• Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective
• An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay
• Using a Logarithmic Mapping to Enable Lower Discount Factors in Reinforcement Learning
• DOM-Q-NET: Grounded RL on Structured Language
• RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems
• Self-Imitation Advantage Learning
• MULEX: Disentangling Exploitation from Exploration in Deep RL
• Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking
• VALAN: Vision and Language Agent Navigation
• Provable Representation Learning for Imitation with Contrastive Fourier Features
• Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learning
• Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
• Generalized Decision Transformer for Offline Hindsight Information Matching
• Catalyst.RL: A Distributed Framework for Reproducible RL Research
• Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods
• Lessons from Contextual Bandit Learning in a Customer Support Bot
• Learning Long-Term Reward Redistribution via Randomized Return Decomposition
• Distributional Reinforcement Learning for Multi-Dimensional Reward Functions
• RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem
• No-Regret Reinforcement Learning with Heavy-Tailed Rewards
• Momentum in Reinforcement Learning
• Approximate Model-Based Shielding for Safe Reinforcement Learning
• Review, Analysis and Design of a Comprehensive Deep Reinforcement Learning Framework
• On Catastrophic Interference in Atari 2600 Games
• Human and Multi-Agent collaboration in a human-MARL teaming framework
• Learning to Represent Action Values as a Hypergraph on the Action Vertices
• Sample Efficient Ensemble Learning with Catalyst.RL
• TDprop: Does Jacobi Preconditioning Help Temporal Difference Learning?
• Non-decreasing Quantile Function Network with Efficient Exploration for Distributional Reinforcement Learning
• Improving Experience Replay with Successor Representation
• Fair treatment allocations in social networks
• ConQUR: Mitigating Delusional Bias in Deep Q-learning
• Benchmarking the Spectrum of Agent Capabilities
• The AI Arena: A Framework for Distributed Multi-Agent Reinforcement Learning
• The Difficulty of Passive Learning in Deep Reinforcement Learning
• Planning and Learning Using Adaptive Entropy Tree Search
• Lineage Evolution Reinforcement Learning
• Ranking Policy Gradient
• Large Batch Experience Replay