Uncertainty Prediction for Machine Learning Models of Material Properties

Abstract

Uncertainty quantification in Artificial Intelligence (AI)-based predictions of material properties is of immense importance for the success and reliability of AI applications in material science. While confidence intervals are commonly reported for machine learning (ML) models, prediction intervals, i.e., the evaluation of the uncertainty on each prediction, are seldomly available. In this work we compare 3 different approaches to obtain such individual uncertainty, testing them on 12 ML-physical properties. Specifically, we investigated using the Quantile loss function, machine learning the prediction intervals directly and using Gaussian Processes. We identify each approachs advantages and disadvantages and end up slightly favoring the modeling of the individual uncertainties directly, as it is the easiest to fit and, in most cases, minimizes over-and under-estimation of the predicted errors. All data for training and testing were taken from the publicly available JARVIS-DFT database, and the codes developed for computing the prediction intervals are available through JARVIS-Tools.

References in corpus (9)

• Scikit-learn: Machine Learning in Python
• The Joint Automated Repository for Various Integrated Simulations (JARVIS) for data-driven materials design
• Physically-informed artificial neural networks for atomistic modeling of materials
• A critical examination of compound stability predictions from machine-learned formation energies
• Deep Neural Networks for Accurate Predictions of Garnet Stability
• Convergence and machine learning predictions of Monkhorst-Pack k-points and plane-wave cut-off in high-throughput DFT calculations
• Machine learning with force-field inspired descriptors for materials: fast screening and mapping energy landscape
• High-Dimensional Materials and Process Optimization using Data-driven Experimental Design with Well-Calibrated Uncertainty Estimates
• Elastic properties of bulk and low-dimensional materials using Van der Waals density functional

Cited by in corpus (3)

• Artificial Intelligence in Material Engineering: A review on applications of AI in Material Engineering
• Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design
• Building Robust Machine Learning Models for Small Chemical Science Data: The Case of Shear Viscosity