Imagination-Augmented Agents for Deep Reinforcement Learning

Abstract

We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to interpret predictions from a learned environment model to construct implicit plans in arbitrary ways, by using the predictions as additional context in deep policy networks. I2As show improved data efficiency, performance, and robustness to model misspecification compared to several baselines.

Cited by in corpus (42)

• Deep Reinforcement Learning: An Overview
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• A Review of Tracking, Prediction and Decision Making Methods for Autonomous Driving
• Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
• Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
• Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing
• A Survey on Reinforcement Learning in Aviation Applications
• Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locations
• InfoBot: Transfer and Exploration via the Information Bottleneck
• How higher goals are constructed and collapse under stress: a hierarchical Bayesian control systems perspective
• Recall Traces: Backtracking Models for Efficient Reinforcement Learning
• Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model
• Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
• World Model as a Graph: Learning Latent Landmarks for Planning
• Deep Intrinsically Motivated Continuous Actor-Critic for Efficient Robotic Visuomotor Skill Learning
• Towards Standardising Reinforcement Learning Approaches for Production Scheduling Problems
• Efficient Intrinsically Motivated Robotic Grasping with Learning-Adaptive Imagination in Latent Space
• Improving Robot Dual-System Motor Learning with Intrinsically Motivated Meta-Control and Latent-Space Experience Imagination
• Probabilistic design of optimal sequential decision-making algorithms in learning and control
• Muesli: Combining Improvements in Policy Optimization
• Deep Residual Reinforcement Learning
• Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks
• Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Models
• Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models
• Variational Inference for Data-Efficient Model Learning in POMDPs
• Learning Manipulation under Physics Constraints with Visual Perception
• AutoEG: Automated Experience Grafting for Off-Policy Deep Reinforcement Learning
• Simulated Mental Imagery for Robotic Task Planning
• Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban
• NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations
• Transfer Learning and Curriculum Learning in Sokoban
• PlotThread: Creating Expressive Storyline Visualizations using Reinforcement Learning
• There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning
• Dynamic Horizon Value Estimation for Model-based Reinforcement Learning
• Self-Consistent Models and Values
• Planning with Expectation Models for Control
• Compressing and Comparing the Generative Spaces of Procedural Content Generators
• Episodic Memory for Learning Subjective-Timescale Models
• Biological Blueprints for Next Generation AI Systems
• Zero Shot Learning on Simulated Robots