Probabilistic model predictive safety certification for learning-based control
arXiv:1906.10417
Abstract
Reinforcement learning (RL) methods have demonstrated their efficiency in simulation environments. However, many applications for which RL offers great potential, such as autonomous driving, are also safety critical and require a certified closed-loop behavior in order to meet safety specifications in the presence of physical constraints. This paper introduces a concept, called probabilistic model predictive safety certification (PMPSC), which can be combined with any RL algorithm and provides provable safety certificates in terms of state and input chance constraints for potentially large-scale systems. The certificate is realized through a stochastic tube that safely connects the current system state with a terminal set of states, that is known to be safe. A novel formulation in terms of a convex receding horizon problem allows a recursively feasible real-time computation of such probabilistic tubes, despite the presence of possibly unbounded disturbances. A design procedure for PMPSC relying on bayesian inference and recent advances in probabilistic set invariance is presented. Using a numerical car simulation, the method and its design procedure are illustrated by enhancing a simple RL algorithm with safety certificates.
References in corpus (5)
- Trust Region Policy Optimization
- Learning human behaviors from motion capture by adversarial imitation
- Differentiable Convex Optimization Layers
- Receding-horizon Stochastic Model Predictive Control with Hard Input Constraints and Joint State Chance Constraints
- Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study