Regularizing Trajectory Optimization with Denoising Autoencoders
arXiv:1903.11981
Abstract
Trajectory optimization using a learned model of the environment is one of the core elements of model-based reinforcement learning. This procedure often suffers from exploiting inaccuracies of the learned model. We propose to regularize trajectory optimization by means of a denoising autoencoder that is trained on the same trajectories as the model of the environment. We show that the proposed regularization leads to improved planning with both gradient-based and gradient-free optimizers. We also demonstrate that using regularized trajectory optimization leads to rapid initial learning in a set of popular motor control tasks, which suggests that the proposed approach can be a useful tool for improving sample efficiency.
arXiv commentNeurIPS 2019
References in corpus (12)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Trust Region Policy Optimization
- Addressing Function Approximation Error in Actor-Critic Methods
- Searching for Activation Functions
- Model-Ensemble Trust-Region Policy Optimization
- Uncertainty-Aware Reinforcement Learning for Collision Avoidance
- Temporal Difference Models: Model-Free Deep RL for Model-Based Control
- Model-Based Reinforcement Learning via Meta-Policy Optimization
- Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
- Regularizing Trajectory Optimization with Denoising Autoencoders
- On the exact relationship between the denoising function and the data distribution
- Improving Model-Based Control and Active Exploration with Reconstruction Uncertainty Optimization