Comparison of neural closure models for discretised PDEs

Abstract

Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: "derivative fitting", "trajectory fitting" with discretise-then-optimise, and "trajectory fitting" with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term.

arXiv comment24 pages and 9 figures. Submitted to Computers and Mathematics with Applications. For associated code, see https://github.com/HugoMelchers/neural-closure-models

References in corpus (8)

• Deep Neural Networks for Data-Driven Turbulence Models
• Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model
• A deep learning enabler for non-intrusive reduced order modeling of fluid flows
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Long-time predictive modeling of nonlinear dynamical systems using neural networks
• Dissecting Neural ODEs
• Embedded training of neural-network sub-grid-scale turbulence models
• Neural Closure Models for Dynamical Systems

Cited by in corpus (5)

• Evolution TANN and the identification of internal variables and evolution equations in solid mechanics
• Neural integration for constitutive equations using small data
• Discretize first, filter next: learning divergence-consistent closure models for large-eddy simulation
• Data-Driven RANS Closures Using a Relative Importance Term Analysis Based Classifier for 2D and 3D Separated Flows
• A new data-driven energy-stable Evolve-Filter-Relax model for turbulent flow simulation