Deep Residual Reinforcement Learning
arXiv:1905.01072
Abstract
We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in the DeepMind Control Suite benchmark. Moreover, we find the residual algorithm an effective approach to the distribution mismatch problem in model-based planning. Compared with the existing TD() method, our residual-based method makes weaker assumptions about the model and yields a greater performance boost.
arXiv commentAAMAS 2020
References in corpus (15)
- Continuous control with deep reinforcement learning
- Addressing Function Approximation Error in Actor-Critic Methods
- Continuous Deep Q-Learning with Model-based Acceleration
- Model-Ensemble Trust-Region Policy Optimization
- Imagination-Augmented Agents for Deep Reinforcement Learning
- Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
- The Predictron: End-To-End Learning and Planning
- Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion
- SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
- Model-Based Reinforcement Learning via Meta-Policy Optimization
- Universal Planning Networks
- Should one compute the Temporal Difference fix point or minimize the Bellman Residual? The unified oblique projection view
- TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning
- A Kernel Loss for Solving the Bellman Equation
- Self-Correcting Models for Model-Based Reinforcement Learning