Learning Control Lyapunov Functions from Counterexamples and Demonstrations

Abstract

We present a technique for learning control Lyapunov-like functions, which are used in turn to synthesize controllers for nonlinear dynamical systems that can stabilize the system, or satisfy specifications such as remaining inside a safe set, or eventually reaching a target set while remaining inside a safe set. The learning framework uses a demonstrator that implements a black-box, untrusted strategy presumed to solve the problem of interest, a learner that poses finitely many queries to the demonstrator to infer a candidate function, and a verifier that checks whether the current candidate is a valid control Lyapunov function. The overall learning framework is iterative, eliminating a set of candidates on each iteration using the counterexamples discovered by the verifier and the demonstrations over these counterexamples. We prove its convergence using ellipsoidal approximation techniques from convex optimization. We also implement this scheme using nonlinear MPC controllers to serve as demonstrators for a set of state and trajectory stabilization problems for nonlinear dynamical systems. We show how the verifier can be constructed efficiently using convex relaxations of the verification problem for polynomial systems to semi-definite programming (SDP) problem instances. Our approach is able to synthesize relatively simple polynomial control Lyapunov functions, and in that process replace the MPC using a guaranteed and computationally less expensive controller.

References in corpus (3)

• Safe Model-based Reinforcement Learning with Stability Guarantees
• Learning Lyapunov (Potential) Functions from Counterexamples and Demonstrations
• Safe Learning of Quadrotor Dynamics Using Barrier Certificates

Cited by in corpus (12)

• Automated and Sound Synthesis of Lyapunov Functions with SMT Solvers
• Learning Stability Certificates from Data
• Learning Lyapunov Functions for Piecewise Affine Systems with Neural Network Controllers
• Counter-example guided inductive synthesis of control Lyapunov functions for uncertain systems
• Provably-Stable Neural Network-Based Control of Nonlinear Systems
• Learning Safe Neural Network Controllers with Barrier Certificates
• Certifying Lyapunov Stability of Black-Box Nonlinear Systems via Counterexample Guided Synthesis (Extended Version)
• Lyapunov-stable neural-network control
• Continuous Lyapunov Controller and Chaotic Non-linear System Optimization using Deep Machine Learning
• Robustly Learning Regions of Attraction from Fixed Data
• Learning Lyapunov Functions for Hybrid Systems
• Inductive Certificate Synthesis for Control Design