Probably Approximately Correct Vision-Based Planning using Motion Primitives
arXiv:2002.12852
Abstract
This paper presents an approach for learning vision-based planners that provably generalize to novel environments (i.e., environments unseen during training). We leverage the Probably Approximately Correct (PAC)-Bayes framework to obtain an upper bound on the expected cost of policies across all environments. Minimizing the PAC-Bayes upper bound thus trains policies that are accompanied by a certificate of performance on novel environments. The training pipeline we propose provides strong generalization guarantees for deep neural network policies by (a) obtaining a good prior distribution on the space of policies using Evolutionary Strategies (ES) followed by (b) formulating the PAC-Bayes optimization as an efficiently-solvable parametric convex optimization problem. We demonstrate the efficacy of our approach for producing strong generalization guarantees for learned vision-based motion planners through two simulated examples: (1) an Unmanned Aerial Vehicle (UAV) navigating obstacle fields with an onboard vision sensor, and (2) a dynamic quadrupedal robot traversing rough terrains with proprioceptive and exteroceptive sensors.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- End to End Learning for Self-Driving Cars
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Recurrent World Models Facilitate Policy Evolution
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
- A Note on the PAC Bayesian Theorem
- Universal Planning Networks
- Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
- PAC-Bayes with Backprop