Model-based Lookahead Reinforcement Learning
arXiv:1908.06012
Abstract
Model-based Reinforcement Learning (MBRL) allows data-efficient learning which is required in real world applications such as robotics. However, despite the impressive data-efficiency, MBRL does not achieve the final performance of state-of-the-art Model-free Reinforcement Learning (MFRL) methods. We leverage the strengths of both realms and propose an approach that obtains high performance with a small amount of data. In particular, we combine MFRL and Model Predictive Control (MPC). While MFRL's strength in exploration allows us to train a better forward dynamics model for MPC, MPC improves the performance of the MFRL policy by sampling-based planning. The experimental results in standard continuous control benchmarks show that our approach can achieve MFRL`s level of performance while being as data-efficient as MBRL.
References in corpus (6)
- Model-Ensemble Trust-Region Policy Optimization
- Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
- Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control
- Temporal Difference Models: Model-Free Deep RL for Model-Based Control
- Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion
- Model-Based Reinforcement Learning via Meta-Policy Optimization