Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models

Abstract

Steering a system towards a desired target in a very short amount of time is challenging from a computational standpoint. Indeed, the intrinsically iterative nature of optimal control problems requires multiple simulations of the physical system to be controlled. Moreover, the control action needs to be updated whenever the underlying scenario undergoes variations. Full-order models based on, e.g., the Finite Element Method, do not meet these requirements due to the computational burden they usually entail. On the other hand, conventional reduced order modeling techniques such as the Reduced Basis method, are intrusive, rely on a linear superimposition of modes, and lack of efficiency when addressing nonlinear time-dependent dynamics. In this work, we propose a non-intrusive Deep Learning-based Reduced Order Modeling (DL-ROM) technique for the rapid control of systems described in terms of parametrized PDEs in multiple scenarios. In particular, optimal full-order snapshots are generated and properly reduced by either Proper Orthogonal Decomposition or deep autoencoders (or a combination thereof) while feedforward neural networks are exploited to learn the map from scenario parameters to reduced optimal solutions. Nonlinear dimensionality reduction therefore allows us to consider state variables and control actions that are both low-dimensional and distributed. After (i) data generation, (ii) dimensionality reduction, and (iii) neural networks training in the offline phase, optimal control strategies can be rapidly retrieved in an online phase for any scenario of interest. The computational speedup and the high accuracy obtained with the proposed approach are assessed on different PDE-constrained optimization problems, ranging from the minimization of energy dissipation in incompressible flows modelled through Navier-Stokes equations to the thermal active cooling in heat transfer.

References in corpus (25)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• Koopman operator-based model reduction for switched-system control of PDEs
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Deep Reinforcement Learning for Swarm Systems
• Physics-Informed Neural Nets for Control of Dynamical Systems
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Deep Model Predictive Control with Online Learning for Complex Physical Systems
• Control of chaotic systems by Deep Reinforcement Learning
• Sparse Identification of Nonlinear Dynamics with Control (SINDYc)
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Fast active thermal cloaking through PDE-constrained optimization and reduced-order modeling
• Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators
• On the Universal Transformation of Data-Driven Models to Control Systems
• Flow control by a hybrid use of machine learning and control theory
• Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality
• Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies
• Robust optimal density control of robotic swarms
• Navigation in a simplified Urban Flow through Deep Reinforcement Learning