Reliable Prediction Intervals with Regression Neural Networks

Abstract

This paper proposes an extension to conventional regression Neural Networks (NNs) for replacing the point predictions they produce with prediction intervals that satisfy a required level of confidence. Our approach follows a novel machine learning framework, called Conformal Prediction (CP), for assigning reliable confidence measures to predictions without assuming anything more than that the data are independent and identically distributed (i.i.d.). We evaluate the proposed method on four benchmark datasets and on the problem of predicting Total Electron Content (TEC), which is an important parameter in trans-ionospheric links; for the latter we use a dataset of more than 60000 TEC measurements collected over a period of 11 years. Our experimental results show that the prediction intervals produced by our method are both well-calibrated and tight enough to be useful in practice.

Cited by in corpus (14)

• Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
• Conformal Prediction: a Unified Review of Theory and New Challenges
• Conformalized Quantile Regression
• Guaranteed Coverage Prediction Intervals with Gaussian Process Regression
• Predictive Inference Is Free with the Jackknife+-after-Bootstrap
• Concepts and Applications of Conformal Prediction in Computational Drug Discovery
• Reliable Probabilistic Classification with Neural Networks
• A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecasting
• Improved conformalized quantile regression
• Deep Evidential Regression
• Assurance Monitoring of Cyber-Physical Systems with Machine Learning Components
• Assurance Monitoring of Learning Enabled Cyber-Physical Systems Using Inductive Conformal Prediction based on Distance Learning
• Vesicoureteral Reflux Detection with Reliable Probabilistic Outputs
• AutoCP: Automated Pipelines for Accurate Prediction Intervals