Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Abstract

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed, which we will make available as open-source library of all codes included in this framework.

References in corpus (21)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• DeepXDE: A deep learning library for solving differential equations
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Integrating Machine Learning and Multiscale Modeling: Perspectives, Challenges, and Opportunities in the Biological, Biomedical, and Behavioral Sciences
• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• Multi-fidelity Bayesian Neural Networks: Algorithms and Applications
• Deep Learning: A Bayesian Perspective
• Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms
• Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• Valid prediction intervals for regression problems
• On the Expressiveness of Approximate Inference in Bayesian Neural Networks
• GINNs: Graph-Informed Neural Networks for Multiscale Physics
• Multi-fidelity Generative Deep Learning Turbulent Flows
• A Survey of Bayesian Statistical Approaches for Big Data
• Learning under Model Misspecification: Applications to Variational and Ensemble methods

Cited by in corpus (44)

• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review
• Reliable extrapolation of deep neural operators informed by physics or sparse observations
• Solution of Physics-based Bayesian Inverse Problems with Deep Generative Priors
• Adaptive weighting of Bayesian physics informed neural networks for multitask and multiscale forward and inverse problems
• Bayesian Physics-Informed Neural Networks for the Subsurface Tomography based on the Eikonal Equation
• Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors
• Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
• Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search
• Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models
• Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference
• Data-driven modelling of brain activity using neural networks, Diffusion Maps, and the Koopman operator
• A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes
• Real-Time Prediction of Gas Flow Dynamics in Diesel Engines using a Deep Neural Operator Framework
• L-HYDRA: Multi-Head Physics-Informed Neural Networks
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• Applications of emulation and Bayesian methods in heavy-ion physics
• Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• Accelerated Bayesian Inference for Molecular Simulations using Local Gaussian Process Surrogate Models
• A Robust Learning Methodology for Uncertainty-aware Scientific Machine Learning models
• Forecasting VIX using Bayesian Deep Learning
• Generalised likelihood profiles for models with intractable likelihoods
• A physics-informed neural network method for the approximation of slow invariant manifolds for the general class of stiff systems of ODEs
• Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks
• Scientific machine learning in Hydrology: a unified perspective
• A Probabilistic Model for Aircraft in Climb using Monotonic Functional Gaussian Process Emulators
• Uncertainty Quantification of Surrogate Models using Conformal Prediction
• Auto-weighted Bayesian Physics-Informed Neural Networks and robust estimations for multitask inverse problems in pore-scale imaging of dissolution
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• A generative adversarial network optimization method for damage detection and digital twinning by deep AI fault learning: Z24 Bridge structural health monitoring benchmark validation
• Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
• Combined Data and Deep Learning Model Uncertainties: An Application to the Measurement of Solid Fuel Regression Rate
• Accelerated Gradient-based Design Optimization Via Differentiable Physics-Informed Neural Operator: A Composites Autoclave Processing Case Study
• Evaluating the Quality of the Quantified Uncertainty for (Re)Calibration of Data-Driven Regression Models
• Fredholm Neural Networks
• Uncertainty quantification and stability of neural operators for prediction of three-dimensional turbulence
• Bayesian Reasoning for Physics Informed Neural Networks
• Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification
• Physics-Informed Machine Learning for Transformer Condition Monitoring -- Part II: Physics-Informed Neural Networks and Uncertainty Quantification
• Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem
• Spatio-Temporal Uncertainty-Modulated Physics-Informed Neural Networks for Solving Hyperbolic Conservation Laws with Strong Shocks
• Uncertainty in AI-driven Monte Carlo simulations
• Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Survey of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems