Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

Abstract

Accurate uncertainty quantification is necessary to enhance the reliability of deep learning models in real-world applications. In the case of regression tasks, prediction intervals (PIs) should be provided along with the deterministic predictions of deep learning models. Such PIs are useful or "high-quality" as long as they are sufficiently narrow and capture most of the probability density. In this paper, we present a method to learn prediction intervals for regression-based neural networks automatically in addition to the conventional target predictions. In particular, we train two companion neural networks: one that uses one output, the target estimate, and another that uses two outputs, the upper and lower bounds of the corresponding PI. Our main contribution is the design of a novel loss function for the PI-generation network that takes into account the output of the target-estimation network and has two optimization objectives: minimizing the mean prediction interval width and ensuring the PI integrity using constraints that maximize the prediction interval probability coverage implicitly. Furthermore, we introduce a self-adaptive coefficient that balances both objectives within the loss function, which alleviates the task of fine-tuning. Experiments using a synthetic dataset, eight benchmark datasets, and a real-world crop yield prediction dataset showed that our method was able to maintain a nominal probability coverage and produce significantly narrower PIs without detriment to its target estimation accuracy when compared to those PIs generated by three state-of-the-art neural-network-based methods. In other words, our method was shown to produce higher-quality PIs.

Accepted at the IEEE Transactions on Neural Networks and Learning Systems

References in corpus (8)

• Variational Inference: A Review for Statisticians
• Ensemble deep learning: A review
• Deep and Confident Prediction for Time Series at Uber
• Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem
• The Two-Piece Normal, Binormal, or Double Gaussian Distribution: Its Origin and Rediscoveries
• Quality of Uncertainty Quantification for Bayesian Neural Network Inference
• Plex: Towards Reliability using Pretrained Large Model Extensions
• PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction