Attention-Enhanced Neural Network Models for Turbulence Simulation

Abstract

Deep neural network models have shown a great potential in accelerating the simulation of fluid dynamic systems. Once trained, these models can make inference within seconds, thus can be extremely efficient. However, they suffer from a generalization problem when the flow becomes chaotic and turbulent. One of the most important reasons is that, existing models lack the mechanism to handle the unique characteristic of turbulent flow: multi-scale flow structures are non-uniformly distributed and strongly nonequilibrium. In this work, we address this issue with the concept of visual attention: intuitively, we expect the attention module to capture the nonequilibrium of turbulence by automatically adjusting weights on different regions. We benchmark the performance improvement with a state of the art neural network model, the Fourier Neural Operator (FNO), on two-dimensional (2D) turbulence prediction task. Numerical experiments show that the attention-enhanced neural network model can generalize well on higher Reynolds numbers flow, and can accurately reconstruct a variety of statistics and instantaneous spatial structures of turbulence. The attention mechanism provides 40% error reduction with 1% increase of parameters, at the same level of computational cost.

arXiv comment18 pages, 25 figures

References in corpus (26)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Machine Learning for Fluid Mechanics
• Self-Attention Generative Adversarial Networks
• DGM: A deep learning algorithm for solving partial differential equations
• Turbulence Modeling in the Age of Data
• Deep learning for universal linear embeddings of nonlinear dynamics
• Fourier Neural Operator for Parametric Partial Differential Equations
• Machine learning accelerated computational fluid dynamics
• Deep Neural Networks for Data-Driven Turbulence Models
• Sub-grid modelling for two-dimensional turbulence using neural networks
• A new prediction method of unsteady wake flow by the hybrid deep neural network
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• Graph neural networks for laminar flow prediction around random 2D shapes
• High-fidelity reconstruction of turbulent flow from spatially limited data using enhanced super-resolution generative adversarial network
• Explore missing flow dynamics by physics-informed deep learning: the parameterised governing systems
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• Deep-learning of Parametric Partial Differential Equations from Sparse and Noisy Data
• Model fusion with physics-guided machine learning
• BCR-Net: a neural network based on the nonstandard wavelet form
• Interpreting neural network models of residual scalar flux
• U-net architectures for fast prediction of incompressible laminar flows
• NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data
• Machine-learning accelerated turbulence modelling of transient flashing jets
• An exploratory study on machine learning to couple numerical solutions of partial differential equations

Cited by in corpus (9)

• Deep Learning in Deterministic Computational Mechanics
• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• A Mathematical Guide to Operator Learning
• Investigation of nonlocal data-driven methods for subgrid-scale stress modelling in large eddy simulation
• Magnetohydrodynamics with Physics Informed Neural Operators
• Machine learning for phase-resolved reconstruction of nonlinear ocean wave surface elevations from sparse remote sensing data
• Machine-learning-based simulation of turbulent flows over periodic hills using a hybrid U-Net and Fourier neural operator framework
• Evaluation of Deep Neural Operator Models toward Ocean Forecasting