Towards a Simple Approach to Multi-step Model-based Reinforcement Learning

Abstract

When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn a single-step transition model. In this paper, we propose a multi-step model that predicts the outcome of an action sequence with variable length. We show that this model is easy to learn, and that the model can make policy-conditional predictions. We report preliminary results that show a clear advantage for the multi-step model compared to its one-step counterpart.

References in corpus (8)

• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
• The Predictron: End-To-End Learning and Planning
• Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping
• Lipschitz Continuity in Model-based Reinforcement Learning
• Mean Actor Critic
• Policy Error Bounds for Model-Based Reinforcement Learning with Factored Linear Models
• Equivalence Between Wasserstein and Value-Aware Loss for Model-based Reinforcement Learning