Stability-certified reinforcement learning: A control-theoretic perspective

Abstract

We investigate the important problem of certifying stability of reinforcement learning policies when interconnected with nonlinear dynamical systems. We show that by regulating the input-output gradients of policies, strong guarantees of robust stability can be obtained based on a proposed semidefinite programming feasibility problem. The method is able to certify a large set of stabilizing controllers by exploiting problem-specific structures; furthermore, we analyze and establish its (non)conservatism. Empirical evaluations on two decentralized control tasks, namely multi-flight formation and power system frequency regulation, demonstrate that the reinforcement learning agents can have high performance within the stability-certified parameter space, and also exhibit stable learning behaviors in the long run.

References in corpus (2)

• A Berkeley View of Systems Challenges for AI
• Constrained Policy Optimization

Cited by in corpus (9)

• Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks
• A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines
• Lipschitz constant estimation of Neural Networks via sparse polynomial optimization
• Learning Lyapunov Functions for Piecewise Affine Systems with Neural Network Controllers
• Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers
• Verification of Neural Network Control Policy Under Persistent Adversarial Perturbation
• Enforcing robust control guarantees within neural network policies
• Imitation Learning with Stability and Safety Guarantees
• Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimization