Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models

Abstract

Continuous monitoring and real-time control of high-dimensional distributed systems are often crucial in applications to ensure a desired physical behavior, without degrading stability and system performances. Traditional feedback control design that relies on full-order models, such as high-dimensional state-space representations or partial differential equations, fails to meet these requirements due to the delay in the control computation, which requires multiple expensive simulations of the physical system. The computational bottleneck is even more severe when considering parametrized systems, as new strategies have to be determined for every new scenario. To address these challenges, we propose a real-time closed-loop control strategy enhanced by nonlinear non-intrusive Deep Learning-based Reduced Order Models (DL-ROMs). Specifically, in the offline phase, (i) full-order state-control pairs are generated for different scenarios through the adjoint method, (ii) the essential features relevant for control design are extracted from the snapshots through a combination of Proper Orthogonal Decomposition (POD) and deep autoencoders, and (iii) the low-dimensional policy bridging latent control and state spaces is approximated with a feedforward neural network. After data generation and neural networks training, the optimal control actions are retrieved in real-time for any observed state and scenario. In addition, the dynamics may be approximated through a cheap surrogate model in order to close the loop at the latent level, thus continuously controlling the system in real-time even when full-order state measurements are missing. The effectiveness of the proposed method, in terms of computational speed, accuracy, and robustness against noisy data, is finally assessed on two different high-dimensional optimal transport problems, one of which also involving an underlying fluid flow.

References in corpus (25)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• Koopman operator-based model reduction for switched-system control of PDEs
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Deep Reinforcement Learning for Swarm Systems
• Physics-Informed Neural Nets for Control of Dynamical Systems
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Deep Model Predictive Control with Online Learning for Complex Physical Systems
• Control of chaotic systems by Deep Reinforcement Learning
• Sparse Identification of Nonlinear Dynamics with Control (SINDYc)
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Fast active thermal cloaking through PDE-constrained optimization and reduced-order modeling
• Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators
• On the Universal Transformation of Data-Driven Models to Control Systems
• Flow control by a hybrid use of machine learning and control theory
• Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality
• Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies
• Robust optimal density control of robotic swarms
• Navigation in a simplified Urban Flow through Deep Reinforcement Learning