Deep learning observables in computational fluid dynamics

Abstract

Many large scale problems in computational fluid dynamics such as uncertainty quantification, Bayesian inversion, data assimilation and PDE constrained optimization are considered very challenging computationally as they require a large number of expensive (forward) numerical solutions of the corresponding PDEs. We propose a machine learning algorithm, based on deep artificial neural networks, that predicts the underlying \emph{input parameters to observable} map from a few training samples (computed realizations of this map). By a judicious combination of theoretical arguments and empirical observations, we find suitable network architectures and training hyperparameters that result in robust and efficient neural network approximations of the parameters to observable map. Numerical experiments are presented to demonstrate low prediction errors for the trained network networks, even when the network has been trained with a few samples, at a computational cost which is several orders of magnitude lower than the underlying PDE solver. Moreover, we combine the proposed deep learning algorithm with Monte Carlo (MC) and Quasi-Monte Carlo (QMC) methods to efficiently compute uncertainty propagation for nonlinear PDEs. Under the assumption that the underlying neural networks generalize well, we prove that the deep learning MC and QMC algorithms are guaranteed to be faster than the baseline (quasi-) Monte Carlo methods. Numerical experiments demonstrating one to two orders of magnitude speed up over baseline QMC and MC algorithms, for the intricate problem of computing probability distributions of the observable, are also presented.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• An overview of gradient descent optimization algorithms
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• Solving high-dimensional partial differential equations using deep learning
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks
• Construction of approximate entropy measure valued solutions for hyperbolic systems of conservation laws
• A Priori Estimates of the Population Risk for Two-layer Neural Networks
• An Efficient Deep Learning Technique for the Navier-Stokes Equations: Application to Unsteady Wake Flow Dynamics
• Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
• A machine learning framework for data driven acceleration of computations of differential equations

Cited by in corpus (39)

• Physics Informed Neural Networks for Simulating Radiative Transfer
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Deep splitting method for parabolic PDEs
• Discontinuity Computing using Physics-Informed Neural Network
• An overview on deep learning-based approximation methods for partial differential equations
• Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
• Integrated Finite Element Neural Network (I-FENN) for non-local continuum damage mechanics
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• Graph Convolutional Neural Networks for Body Force Prediction
• Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number
• Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations
• Numerical simulations for full history recursive multilevel Picard approximations for systems of high-dimensional partial differential equations
• On universal approximation and error bounds for Fourier Neural Operators
• Uniform error estimates for artificial neural network approximations for heat equations
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach
• Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities
• Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs
• Unveiling the frontiers of deep learning: innovations shaping diverse domains
• Statistical solutions of the incompressible Euler equations
• Deep neural network approximations for Monte Carlo algorithms
• A graph-based probabilistic geometric deep learning framework with online enforcement of physical constraints to predict the criticality of defects in porous materials
• Generalised multilevel Picard approximations
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Multilevel Picard approximations for high-dimensional semilinear second-order PDEs with Lipschitz nonlinearities
• Neural Networks for Nuclear Reactions in MAESTROeX
• Cell-average based neural network method for hyperbolic and parabolic partial differential equations
• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• An operator learning perspective on parameter-to-observable maps
• Higher-order Quasi-Monte Carlo Training of Deep Neural Networks
• On the approximation of functions by tanh neural networks
• Real-time simulation of parameter-dependent fluid flows through deep learning-based reduced order models
• Model Reduction for Advection Dominated Hyperbolic Problems in an ALE Framework: Offline and Online Phases
• Looking at the posterior: accuracy and uncertainty of neural-network predictions
• labrador: A domain-optimized machine-learning tool for gravitational wave inference
• Complete CVDL Methodology for Investigating Hydrodynamic Instabilities
• Constructing Gradient Controllable Recurrent Neural Networks Using Hamiltonian Dynamics
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs