A robust and stable hybrid neural network/finite element method for 2D flows that generalizes to different geometries

Abstract

The deep neural network multigrid solver (DNN-MG) combines a coarse-grid finite element simulation with a deep neural network that corrects the solution on finer grid levels, thereby improving the computational efficiency. In this work, we discuss various design choices for the DNN-MG method and demonstrate significant improvements in accuracy and generalizability when applied to the solution of the nonstationary Navier-Stokes equations. We investigate the stability of the hybrid simulation and show how the neural networks can be made more robust with the help of replay buffers. After an initial single-step training, we run the hybrid simulation for extended periods and compute new reference solutions of the neural network perturbed state for each step. By retraining on this data, the error caused by the neural network over multiple time-steps due to distributional shift can be effectively reduced without the need for a differentiable numerical solver. Furthermore, we compare multiple neural network architectures, including recurrent neural networks and Transformers, and study their ability to utilize more information from an increased temporal and spatial receptive field. Transformers allow us to make use of information from cells outside the predicted patch even with unstructured meshes while maintaining the locality of our approach. This can further improve the accuracy of DNN-MG without a significant impact on performance.

References in corpus (13)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks
• Neural General Circulation Models for Weather and Climate
• A neural network multigrid solver for the Navier-Stokes equations
• FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead
• Machine learning for online sea ice bias correction within global ice-ocean simulations
• Accelerating high order discontinuous Galerkin solvers using neural networks: 3D compressible Navier-Stokes equations
• DNN-MG: A Hybrid Neural Network/Finite Element Method with Applications to 3D Simulations of the Navier-Stokes Equations
• Second order pressure estimates for the Crank-Nicolson discretization of the incompressible Navier-Stokes Equations
• A Newton multigrid framework for optimal control of fluid-structure interactions
• Structure Preservation for the Deep Neural Network Multigrid Solver
• A hybrid finite element/neural network solver and its application to the Poisson problem