Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning

Abstract

There is a growing interest in developing data-driven subgrid-scale (SGS) models for large-eddy simulation (LES) using machine learning (ML). In a priori (offline) tests, some recent studies have found ML-based data-driven SGS models that are trained on high-fidelity data (e.g., from direct numerical simulation, DNS) to outperform baseline physics-based models and accurately capture the inter-scale transfers, both forward (diffusion) and backscatter. While promising, instabilities in a posteriori (online) tests and inabilities to generalize to a different flow (e.g., with a higher Reynolds number, Re) remain as major obstacles in broadening the applications of such data-driven SGS models. For example, many of the same aforementioned studies have found instabilities that required often ad-hoc remedies to stabilize the LES at the expense of reducing accuracy. Here, using 2D decaying turbulence as the testbed, we show that deep fully convolutional neural networks (CNNs) can accurately predict the SGS forcing terms and the inter-scale transfers in a priori tests, and if trained with enough samples, lead to stable and accurate a posteriori LES-CNN. Further analysis attributes these instabilities to the disproportionally lower accuracy of the CNNs in capturing backscattering when the training set is small. We also show that transfer learning, which involves re-training the CNN with a small amount of data (e.g., 1%) from the new flow, enables accurate and stable a posteriori LES-CNN for flows with 16x higher Re (as well as higher grid resolution if needed). These results show the promise of CNNs with transfer learning to provide stable, accurate, and generalizable LES for practical use.

arXiv comment30 pages, 12 figures

References in corpus (37)

• How transferable are features in deep neural networks?
• Machine Learning for Fluid Mechanics
• Turbulence Modeling in the Age of Data
• Deep learning to represent sub-grid processes in climate models
• A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
• Learning data driven discretizations for partial differential equations
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• Deep Neural Networks for Data-Driven Turbulence Models
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Sub-grid modelling for two-dimensional turbulence using neural networks
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Searching for turbulence models by artificial neural network
• Data-assisted reduced-order modeling of extreme events in complex dynamical systems
• Analog forecasting of extreme-causing weather patterns using deep learning
• A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• An update on the double cascade scenario in two-dimensional turbulence
• Probabilistic neural networks for fluid flow surrogate modeling and data recovery
• Data-Driven Modelling of the Reynolds Stress Tensor using Random Forests with Invariance
• Study of dynamics in post-transient flows using Koopman mode decomposition
• Long-time predictive modeling of nonlinear dynamical systems using neural networks
• Calibration and Uncertainty Quantification of Convective Parameters in an Idealized GCM
• Multi-level Dynamical Systems: Connecting the Ruelle Response Theory and the Mori-Zwanzig Approach
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence
• A Dynamic Subgrid Scale Model for Large Eddy Simulations Based on the Mori-Zwanzig Formalism
• Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning
• The Universal Aspect Ratio of Vortices in Rotating Stratified Flows: Theory and Simulation
• Deep learning of mixing by two 'atoms' of stratified turbulence
• Interpreting neural network models of residual scalar flux
• A 3D Spectral Anelastic Hydrodynamic Code for Shearing, Stratified Flows
• Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulations
• Interface learning in fluid dynamics: statistical inference of closures within micro-macro coupling models
• Reduced-order modeling of fully turbulent buoyancy-driven flows using the Green's function method
• Artificial neural network approach for turbulence models: A local framework

Cited by in corpus (24)

• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• A posteriori learning for quasi-geostrophic turbulence parametrization
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• Deep Learning Closure Models for Large-Eddy Simulation of Flows around Bluff Bodies
• Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers
• Data-driven nonlinear turbulent flow scaling with Buckingham Pi variables
• Generative data-driven approaches for stochastic subgrid parameterizations in an idealized ocean model
• Investigation of nonlocal data-driven methods for subgrid-scale stress modelling in large eddy simulation
• Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges
• Compressing fluid flows with nonlinear machine learning: mode decomposition, latent modeling, and flow control
• A Multifidelity deep operator network approach to closure for multiscale systems
• Revisiting Tensor Basis Neural Networks for Reynolds stress modeling: application to plane channel and square duct flows
• A stable implementation of a data-driven scale-aware mesoscale parameterization
• A recursive neural-network-based subgrid-scale model for large eddy simulation: application to homogeneous isotropic turbulence
• Subgrid parameterizations of ocean mesoscale eddies based on Germano decomposition
• Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case
• Adjoint-based variational optimal mixed models for large-eddy simulation of turbulence
• Reduced Data-Driven Turbulence Closure for Capturing Long-Term Statistics
• Rotating shallow water equations with bottom drag: bifurcations and growth due to kinetic energy backscatter
• Neural network-based Godunov corrections for approximate Riemann solvers using bi-fidelity learning
• Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence
• A new data-driven energy-stable Evolve-Filter-Relax model for turbulent flow simulation
• Evaluation of Analytical Turbulence Closures for Quasi-Geostrophic Ocean Flows with Coastal Boundaries