The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems

Abstract

Learning algorithms have shown considerable prowess in simulation by allowing robots to adapt to uncertain environments and improve their performance. However, such algorithms are rarely used in practice on safety-critical systems, since the learned policy typically does not yield any safety guarantees. That is, the required exploration may cause physical harm to the robot or its environment. In this paper, we present a method to learn accurate safety certificates for nonlinear, closed-loop dynamical systems. Specifically, we construct a neural network Lyapunov function and a training algorithm that adapts it to the shape of the largest safe region in the state space. The algorithm relies only on knowledge of inputs and outputs of the dynamics, rather than on any specific model structure. We demonstrate our method by learning the safe region of attraction for a simulated inverted pendulum. Furthermore, we discuss how our method can be used in safe learning algorithms together with statistical models of dynamical systems.

Proc. of the 2nd Conference on Robot Learning (CoRL 2018)

Cited by in corpus (26)

• Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overview
• Formal Synthesis of Lyapunov Neural Networks
• Learning for Safety-Critical Control with Control Barrier Functions
• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Active Learning in Robotics: A Review of Control Principles
• Neural Contraction Metrics for Robust Estimation and Control: A Convex Optimization Approach
• Learning Stable Deep Dynamics Models
• Learning Stability Certificates from Data
• Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions
• Lyapunov-Based Reinforcement Learning State Estimator
• Safe Interactive Model-Based Learning
• Almost Surely Stable Deep Dynamics
• Promoting global stability in data-driven models of quadratic nonlinear dynamics
• Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations
• Distributionally robust risk map for learning-based motion planning and control: A semidefinite programming approach
• MANGA: Method Agnostic Neural-policy Generalization and Adaptation
• Neural Lyapunov Redesign
• Lyapunov-stable neural-network control
• Continuous Lyapunov Controller and Chaotic Non-linear System Optimization using Deep Machine Learning
• LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Tasks
• Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples
• Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics
• Synthesis of Feedback Controller for Nonlinear Control Systems with Optimal Region of Attraction
• Learning Region of Attraction for Nonlinear Systems
• Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation
• FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimize