System stabilization with policy optimization on unstable latent manifolds

Abstract

Stability is a basic requirement when studying the behavior of dynamical systems. However, stabilizing dynamical systems via reinforcement learning is challenging because only little data can be collected over short time horizons before instabilities are triggered and data become meaningless. This work introduces a reinforcement learning approach that is formulated over latent manifolds of unstable dynamics so that stabilizing policies can be trained from few data samples. The unstable manifolds are minimal in the sense that they contain the lowest dimensional dynamics that are necessary for learning policies that guarantee stabilization. This is in stark contrast to generic latent manifolds that aim to approximate all -- stable and unstable -- system dynamics and thus are higher dimensional and often require higher amounts of data. Experiments demonstrate that the proposed approach stabilizes even complex physical systems from few data samples for which other methods that operate either directly in the system state space or on generic latent manifolds fail.

29 pages, 10 figures, 1 table

References in corpus (17)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
• On Dynamic Mode Decomposition: Theory and Applications
• Model-free control
• Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control
• Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
• A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder
• Nonlinear Model Order Reduction via Lifting Transformations and Proper Orthogonal Decomposition
• Operator inference for non-intrusive model reduction with quadratic manifolds
• Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques
• Data-driven Structured Realization
• Robust output-feedback stabilization for incompressible flows using low-dimensional ${\mathcal{H}}_{\infty }$-controllers
• The numerical computation of unstable manifolds for infinite dimensional dynamical systems by embedding techniques
• On the sample complexity of stabilizing linear dynamical systems from data
• Context-aware controller inference for stabilizing dynamical systems from scarce data
• Multi-fidelity robust controller design with gradient sampling
• CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations