Stein Variational Model Predictive Control
arXiv:2011.07641
Abstract
Decision making under uncertainty is critical to real-world, autonomous systems. Model Predictive Control (MPC) methods have demonstrated favorable performance in practice, but remain limited when dealing with complex probability distributions. In this paper, we propose a generalization of MPC that represents a multitude of solutions as posterior distributions. By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem. We present a Stein variational gradient descent method to estimate the posterior directly over control parameters, given a cost function and observed state trajectories. We show that this framework leads to successful planning in challenging, non-convex optimal control problems.
Accepted to Conference on Robot Learning (CoRL) 2020
References in corpus (12)
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
- Path integrals and symmetry breaking for optimal control theory
- A linear theory for control of non-linear stochastic systems
- Optimal control as a graphical model inference problem
- Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference
- Stein Variational Policy Gradient
- Message Passing Stein Variational Gradient Descent
- Stein Point Markov Chain Monte Carlo
- Approximate Inference and Stochastic Optimal Control
- Real-Time Stochastic Optimal Control for Multi-agent Quadrotor Systems
- Stochastic Optimal Control in Continuous Space-Time Multi-Agent Systems
- A Robust Closed-Loop Biped Locomotion Planner Based on Time Varying Model Predictive Control