Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems

Abstract

Simulating complex physical systems often involves solving partial differential equations (PDEs) with some closures due to the presence of multi-scale physics that cannot be fully resolved. Therefore, reliable and accurate closure models for unresolved physics remains an important requirement for many computational physics problems, e.g., turbulence simulation. Recently, several researchers have adopted generative adversarial networks (GANs), a novel paradigm of training machine learning models, to generate solutions of PDEs-governed complex systems without having to numerically solve these PDEs. However, GANs are known to be difficult in training and likely to converge to local minima, where the generated samples do not capture the true statistics of the training data. In this work, we present a statistical constrained generative adversarial network by enforcing constraints of covariance from the training data, which results in an improved machine-learning-based emulator to capture the statistics of the training data generated by solving fully resolved PDEs. We show that such a statistical regularization leads to better performance compared to standard GANs, measured by (1) the constrained model's ability to more faithfully emulate certain physical properties of the system and (2) the significantly reduced (by up to 80%) training time to reach the solution. We exemplify this approach on the Rayleigh-Benard convection, a turbulent flow system that is an idealized model of the Earth's atmosphere. With the growth of high-fidelity simulation databases of physical systems, this work suggests great potential for being an alternative to the explicit modeling of closures or parameterizations for unresolved physics, which are known to be a major source of uncertainty in simulating multi-scale physical systems, e.g., turbulence or Earth's climate.

References in corpus (24)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• Deep learning for universal linear embeddings of nonlinear dynamics
• Deep learning to represent sub-grid processes in climate models
• A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• Boundary layer structure in turbulent thermal convection and its consequences for the required numerical resolution
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
• On Convergence and Stability of GANs
• Turbulent superstructures in Rayleigh-Bénard convection
• A Comparison of Turbulent Thermal Convection Between Conditions of Constant Temperature and Constant Flux
• Comparison between two and three dimensional Rayleigh-Bénard convection
• Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
• Dynamics of reorientations and reversals of large scale flow in Rayleigh-Benard convection
• Lat-Net: Compressing Lattice Boltzmann Flow Simulations using Deep Neural Networks
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks
• Evolutionary Generative Adversarial Networks
• Emergence of skewed non-Gaussian distributions of velocity increments in isotropic turbulence
• Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems
• Model Reduction with Memory and the Machine Learning of Dynamical Systems

Cited by in corpus (35)

• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Deep unsupervised learning of turbulence for inflow generation at various Reynolds numbers
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Model fusion with physics-guided machine learning
• A Survey on Physics Informed Reinforcement Learning: Review and Open Problems
• Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning
• Machine Learning for Physics-Informed Generation of Dispersed Multiphase Flow Using Generative Adversarial Networks
• Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence
• Equation-free surrogate modeling of geophysical flows at the intersection of machine learning and data assimilation
• MillimeterDL: Deep Learning Simulations of the Microwave Sky
• Coherent Solutions and Transition to Turbulence in Two-Dimensional Rayleigh-Bénard Convection
• Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems
• Deep Learning of Dynamic Subsurface Flow via Theory-guided Generative Adversarial Network
• Subgrid-scale parametrization of unresolved scales in forced Burgers equation using Generative Adversarial Networks (GAN)
• Neural Network Methods for Radiation Detectors and Imaging
• Data assimilation empowered neural network parameterizations for subgrid processes in geophysical flows
• Generative Adversarial Networks with Physical Evaluators for Spray Simulation of Pintle Injector
• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• DeepClimGAN: A High-Resolution Climate Data Generator
• STAN: Spatio-Temporal Attention Network for Pandemic Prediction Using Real World Evidence
• Physics Encoded Blocks in Residual Neural Network Architectures for Digital Twin Models
• Frame-independent vector-cloud neural network for nonlocal constitutive modeling on arbitrary grids
• Assimilation of disparate data for enhanced reconstruction of turbulent mean flows
• A multiscale and multicriteria Generative Adversarial Network to synthesize 1-dimensional turbulent fields
• Adversarial sampling of unknown and high-dimensional conditional distributions
• Physical Context and Timing Aware Sequence Generating GANs
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System