Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Abstract

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed, which we will make available as open-source library of all codes included in this framework.

References in corpus (4)

• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• Integrating Machine Learning and Multiscale Modeling: Perspectives, Challenges, and Opportunities in the Biological, Biomedical, and Behavioral Sciences
• Deep Learning: A Bayesian Perspective
• A Survey of Bayesian Statistical Approaches for Big Data

Cited by in corpus (43)

• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review
• Reliable extrapolation of deep neural operators informed by physics or sparse observations
• Adaptive weighting of Bayesian physics informed neural networks for multitask and multiscale forward and inverse problems
• Bayesian Physics-Informed Neural Networks for the Subsurface Tomography based on the Eikonal Equation
• Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors
• Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
• Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search
• Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models
• Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference
• Data-driven modelling of brain activity using neural networks, Diffusion Maps, and the Koopman operator
• A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes
• L-HYDRA: Multi-Head Physics-Informed Neural Networks
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• Real-Time Prediction of Gas Flow Dynamics in Diesel Engines using a Deep Neural Operator Framework
• Applications of emulation and Bayesian methods in heavy-ion physics
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning
• Accelerated Bayesian Inference for Molecular Simulations using Local Gaussian Process Surrogate Models
• A Robust Learning Methodology for Uncertainty-aware Scientific Machine Learning models
• Forecasting VIX using Bayesian Deep Learning
• Generalised likelihood profiles for models with intractable likelihoods
• A physics-informed neural network method for the approximation of slow invariant manifolds for the general class of stiff systems of ODEs
• Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks
• Scientific machine learning in Hydrology: a unified perspective
• A Probabilistic Model for Aircraft in Climb using Monotonic Functional Gaussian Process Emulators
• Uncertainty Quantification of Surrogate Models using Conformal Prediction
• Auto-weighted Bayesian Physics-Informed Neural Networks and robust estimations for multitask inverse problems in pore-scale imaging of dissolution
• A generative adversarial network optimization method for damage detection and digital twinning by deep AI fault learning: Z24 Bridge structural health monitoring benchmark validation
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
• Accelerated Gradient-based Design Optimization Via Differentiable Physics-Informed Neural Operator: A Composites Autoclave Processing Case Study
• Fredholm Neural Networks
• Evaluating the Quality of the Quantified Uncertainty for (Re)Calibration of Data-Driven Regression Models
• Combined Data and Deep Learning Model Uncertainties: An Application to the Measurement of Solid Fuel Regression Rate
• Physics-Informed Machine Learning for Transformer Condition Monitoring -- Part II: Physics-Informed Neural Networks and Uncertainty Quantification
• Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem
• Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification
• Bayesian Reasoning for Physics Informed Neural Networks
• Uncertainty quantification and stability of neural operators for prediction of three-dimensional turbulence
• Spatio-Temporal Uncertainty-Modulated Physics-Informed Neural Networks for Solving Hyperbolic Conservation Laws with Strong Shocks
• Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Survey of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems
• Uncertainty in AI-driven Monte Carlo simulations