Safe end-to-end imitation learning for model predictive control

Abstract

We propose the use of Bayesian networks, which provide both a mean value and an uncertainty estimate as output, to enhance the safety of learned control policies under circumstances in which a test-time input differs significantly from the training set. Our algorithm combines reinforcement learning and end-to-end imitation learning to simultaneously learn a control policy as well as a threshold over the predictive uncertainty of the learned model, with no hand-tuning required. Corrective action, such as a return of control to the model predictive controller or human expert, is taken when the uncertainty threshold is exceeded. We validate our method on fully-observable and vision-based partially-observable systems using cart-pole and autonomous driving simulations using deep convolutional Bayesian neural networks. We demonstrate that our method is robust to uncertainty resulting from varying system dynamics as well as from partial state observability.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Improving neural networks by preventing co-adaptation of feature detectors
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Weight Uncertainty in Neural Networks
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• MPC-Inspired Neural Network Policies for Sequential Decision Making
• DropoutDAgger: A Bayesian Approach to Safe Imitation Learning
• Adaptive Probabilistic Trajectory Optimization via Efficient Approximate Inference

Cited by in corpus (8)

• A Survey of Deep RL and IL for Autonomous Driving Policy Learning
• Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations
• On the Sample Complexity of Stability Constrained Imitation Learning
• Iterative Amortized Policy Optimization
• Perceptual Attention-based Predictive Control
• CARPAL: Confidence-Aware Intent Recognition for Parallel Autonomy
• MPC-guided Imitation Learning of Neural Network Policies for the Artificial Pancreas
• Uncertainty-Aware Data Aggregation for Deep Imitation Learning