Natural Environment Benchmarks for Reinforcement Learning

Abstract

While current benchmark reinforcement learning (RL) tasks have been useful to drive progress in the field, they are in many ways poor substitutes for learning with real-world data. By testing increasingly complex RL algorithms on low-complexity simulation environments, we often end up with brittle RL policies that generalize poorly beyond the very specific domain. To combat this, we propose three new families of benchmark RL domains that contain some of the complexity of the natural world, while still supporting fast and extensive data acquisition. The proposed domains also permit a characterization of generalization through fair train/test separation, and easy comparison and replication of results. Through this work, we challenge the RL research community to develop more robust algorithms that meet high standards of evaluation.

arXiv comment12 figures

References in corpus (12)

• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Deep Residual Learning for Image Recognition
• The Kinetics Human Action Video Dataset
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
• Deep Reinforcement Learning for Visual Object Tracking in Videos
• HoME: a Household Multimodal Environment

Cited by in corpus (24)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Leveraging Procedural Generation to Benchmark Reinforcement Learning
• Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO
• Learning Invariant Representations for Reinforcement Learning without Reconstruction
• RecSim: A Configurable Simulation Platform for Recommender Systems
• Invariant Causal Prediction for Block MDPs
• Making Deep Q-learning methods robust to time discretization
• Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
• The Scientific Method in the Science of Machine Learning
• Unsupervised State Representation Learning in Atari
• Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations
• Observational Overfitting in Reinforcement Learning
• The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels
• Rotation, Translation, and Cropping for Zero-Shot Generalization
• Measuring Visual Generalization in Continuous Control from Pixels
• Intervention Design for Effective Sim2Real Transfer
• Domain Adversarial Reinforcement Learning
• Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning
• Procedural Generalization by Planning with Self-Supervised World Models
• Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments
• Proximal Policy Gradient: PPO with Policy Gradient
• DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck
• The Principle of Unchanged Optimality in Reinforcement Learning Generalization
• Learning Representations for Pixel-based Control: What Matters and Why?