Neural Lyapunov Redesign

Abstract

Learning controllers merely based on a performance metric has been proven effective in many physical and non-physical tasks in both control theory and reinforcement learning. However, in practice, the controller must guarantee some notion of safety to ensure that it does not harm either the agent or the environment. Stability is a crucial notion of safety, whose violation can certainly cause unsafe behaviors. Lyapunov functions are effective tools to assess stability in nonlinear dynamical systems. In this paper, we combine an improving Lyapunov function with automatic controller synthesis in an iterative fashion to obtain control policies with large safe regions. We propose a two-player collaborative algorithm that alternates between estimating a Lyapunov function and deriving a controller that gradually enlarges the stability region of the closed-loop system. We provide theoretical results on the class of systems that can be treated with the proposed algorithm and empirically evaluate the effectiveness of our method using an exemplary dynamical system.

arXiv comment27 pages

References in corpus (13)

• Continuous control with deep reinforcement learning
• Hamiltonian Neural Networks
• Safe Model-based Reinforcement Learning with Stability Guarantees
• Safe Controller Optimization for Quadrotors with Gaussian Processes
• The Numerics of GANs
• The Mechanics of n-Player Differentiable Games
• Neural Lyapunov Control
• The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems
• Deep Lyapunov Function: Automatic Stability Analysis for Dynamical Systems
• Deep Nonlinear Non-Gaussian Filtering for Dynamical Systems
• Analysis of Nonautonomous Adversarial Systems
• Kernel-Guided Training of Implicit Generative Models with Stability Guarantees
• A Local Information Criterion for Dynamical Systems