Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial

Abstract

On top of machine learning models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage and the remaining useful life prediction of turbofan engines.

References in corpus (13)

• Popular Ensemble Methods: An Empirical Study
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Artificial Intelligence and Statistics
• Compressive Sampling of Polynomial Chaos Expansions: Convergence Analysis and Sampling Strategies
• Wireframe-Based UI Design Search Through Image Autoencoder
• Physics-Informed and Hybrid Machine Learning in Additive Manufacturing: Application to Fused Filament Fabrication
• The Case for Bayesian Deep Learning
• Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets
• Simulation free reliability analysis: A physics-informed deep learning based approach
• Dynaformer: A Deep Learning Model for Ageing-aware Battery Discharge Prediction
• A stochastic Stein Variational Newton method
• Towards Goal, Feasibility, and Diversity-Oriented Deep Generative Models in Design

Cited by in corpus (6)

• Physics-Informed Machine Learning for Battery Degradation Diagnostics: A Comparison of State-of-the-Art Methods
• Graph Evidential Learning for Anomaly Detection
• A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks
• A review of simulation, measurement techniques, and development in chip thermal design
• Diffusion Model-based Parameter Estimation in Dynamic Power Systems
• Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Survey of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems