The Arcade Learning Environment: An Evaluation Platform for General Agents

Abstract

In this article we introduce the Arcade Learning Environment (ALE): both a challenge problem and a platform and methodology for evaluating the development of general, domain-independent AI technology. ALE provides an interface to hundreds of Atari 2600 game environments, each one different, interesting, and designed to be a challenge for human players. ALE presents significant research challenges for reinforcement learning, model learning, model-based planning, imitation learning, transfer learning, and intrinsic motivation. Most importantly, it provides a rigorous testbed for evaluating and comparing approaches to these problems. We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning. In doing so, we also propose an evaluation methodology made possible by ALE, reporting empirical results on over 55 different games. All of the software, including the benchmark agents, is publicly available.

Cited by in corpus (79)

• Overcoming catastrophic forgetting in neural networks
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Exploration in Deep Reinforcement Learning: A Survey
• A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
• A Review on Deep Learning Techniques for Video Prediction
• State Representation Learning for Control: An Overview
• The Hanabi Challenge: A New Frontier for AI Research
• First return, then explore
• Flow: A Modular Learning Framework for Mixed Autonomy Traffic
• Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
• Fathom: Reference Workloads for Modern Deep Learning Methods
• Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
• Taurus: A Data Plane Architecture for Per-Packet ML
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Learning to grow: control of material self-assembly using evolutionary reinforcement learning
• Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems
• Queueing Network Controls via Deep Reinforcement Learning
• Neurosymbolic Reinforcement Learning and Planning: A Survey
• Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps
• A Search-Based Testing Approach for Deep Reinforcement Learning Agents
• A Benchmark Environment Motivated by Industrial Control Problems
• A Tutorial on Meta-Reinforcement Learning
• Enhancements for Real-Time Monte-Carlo Tree Search in General Video Game Playing
• Derivative-Free Reinforcement Learning: A Review
• DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
• Online Continual Learning on Sequences
• AI Researchers, Video Games Are Your Friends!
• Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning
• Pre-training with Non-expert Human Demonstration for Deep Reinforcement Learning
• Safe Option-Critic: Learning Safety in the Option-Critic Architecture
• Evolutionary reinforcement learning of dynamical large deviations
• Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review
• Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model
• Navigating the Landscape of Multiplayer Games
• TDM: Trustworthy Decision-Making via Interpretability Enhancement
• Improving robot navigation in crowded environments using intrinsic rewards
• Evolutionary Reinforcement Learning via Cooperative Coevolutionary Negatively Correlated Search
• Analysing Results from AI Benchmarks: Key Indicators and How to Obtain Them
• Gegelati: Lightweight Artificial Intelligence through Generic and Evolvable Tangled Program Graphs
• Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation
• A Human Mixed Strategy Approach to Deep Reinforcement Learning
• Interpretable pipelines with evolutionarily optimized modules for RL tasks with visual inputs
• Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning
• Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Control
• Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks
• Extending Environments To Measure Self-Reflection In Reinforcement Learning
• Sim-to-real reinforcement learning applied to end-to-end vehicle control
• Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agents
• Multi-Agent Deep Reinforcement Learning with Human Strategies
• Explore and Explain: Self-supervised Navigation and Recounting
• Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination
• Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning
• Distributional Reinforcement Learning with Unconstrained Monotonic Neural Networks
• Flow-based Intrinsic Curiosity Module
• Conformal Symplectic Optimization for Stable Reinforcement Learning
• Hybrid Self-Attention NEAT: A novel evolutionary approach to improve the NEAT algorithm
• Crawling in Rogue's dungeons with (partitioned) A3C
• Machine Learning for Quantum-Enhanced Gravitational-Wave Observatories
• HiER: Highlight Experience Replay for Boosting Off-Policy Reinforcement Learning Agents
• Domain Adapting Deep Reinforcement Learning for Real-world Speech Emotion Recognition
• Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learning
• Identifying Critical States by the Action-Based Variance of Expected Return
• Pretty darn good control: when are approximate solutions better than approximate models
• Distributional Reinforcement Learning with Ensembles
• Comparing Reinforcement Learning and Human Learning using the Game of Hidden Rules
• AMaze: An intuitive benchmark generator for fast prototyping of generalizable agents
• Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network
• Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning
• LuckyMera: a Modular AI Framework for Building Hybrid NetHack Agents
• Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning
• CaiRL: A High-Performance Reinforcement Learning Environment Toolkit
• Reusability and Transferability of Macro Actions for Reinforcement Learning
• Planning and Learning Using Adaptive Entropy Tree Search
• On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementations
• Themis: An explainable AI-enabled framework for Reinforcement Learning with Human Feedback