Continuous Control for Searching and Planning with a Learned Model
arXiv:2006.07430
Abstract
Decision-making agents with planning capabilities have achieved huge success in the challenging domain like Chess, Shogi, and Go. In an effort to generalize the planning ability to the more general tasks where the environment dynamics are not available to the agent, researchers proposed the MuZero algorithm that can learn the dynamical model through the interactions with the environment. In this paper, we provide a way and the necessary theoretical results to extend the MuZero algorithm to more generalized environments with continuous action space. Through numerical results on two relatively low-dimensional MuJoCo environments, we show the proposed algorithm outperforms the soft actor-critic (SAC) algorithm, a state-of-the-art model-free deep reinforcement learning algorithm.
References in corpus (12)
- Continuous control with deep reinforcement learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Trust Region Policy Optimization
- Prioritized Experience Replay
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
- Recurrent World Models Facilitate Policy Evolution
- Continuous Deep Q-Learning with Model-based Acceleration
- Benchmarking Model-Based Reinforcement Learning
- On-line Policy Improvement using Monte-Carlo Search
- Observe and Look Further: Achieving Consistent Performance on Atari
- Prioritized Sequence Experience Replay
- A0C: Alpha Zero in Continuous Action Space