Calibrated Reliable Regression using Maximum Mean Discrepancy

Abstract

Accurate quantification of uncertainty is crucial for real-world applications of machine learning. However, modern deep neural networks still produce unreliable predictive uncertainty, often yielding over-confident predictions. In this paper, we are concerned with getting well-calibrated predictions in regression tasks. We propose the calibrated regression method using the maximum mean discrepancy by minimizing the kernel embedding measure. Theoretically, the calibration error of our method asymptotically converges to zero when the sample size is large enough. Experiments on non-trivial real datasets show that our method can produce well-calibrated and sharp prediction intervals, which outperforms the related state-of-the-art methods.

arXiv commentAccepted to NeurIPS'2020. Full version with appendix

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• On Calibration of Modern Neural Networks
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Generative Moment Matching Networks
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach
• Function Space Particle Optimization for Bayesian Neural Networks
• Generative Well-intentioned Networks

Cited by in corpus (2)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Classifier Calibration: A survey on how to assess and improve predicted class probabilities