Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification

Abstract

Modern machine learning models with high accuracy are often miscalibrated -- the predicted top probability does not reflect the actual accuracy, and tends to be over-confident. It is commonly believed that such over-confidence is mainly due to over-parametrization, in particular when the model is large enough to memorize the training data and maximize the confidence. In this paper, we show theoretically that over-parametrization is not the only reason for over-confidence. We prove that logistic regression is inherently over-confident, in the realizable, under-parametrized setting where the data is generated from the logistic model, and the sample size is much larger than the number of parameters. Further, this over-confidence happens for general well-specified binary classification problems as long as the activation is symmetric and concave on the positive part. Perhaps surprisingly, we also show that over-confidence is not always the case -- there exists another activation function (and a suitable loss function) under which the learned classifier is under-confident at some probability values. Overall, our theory provides a precise characterization of calibration in realizable binary classification, which we verify on simulations and real data experiments.

arXiv commentAppearing at ICML 2021

References in corpus (19)

• Message Passing Algorithms for Compressed Sensing
• Wide Residual Networks
• On Calibration of Modern Neural Networks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• A modern maximum-likelihood theory for high-dimensional logistic regression
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• A framework to characterize performance of LASSO algorithms
• Verified Uncertainty Calibration
• Calibrating Deep Neural Networks using Focal Loss
• Asymptotic behavior of unregularized and ridge-regularized high-dimensional robust regression estimators : rigorous results
• Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control
• Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning
• Hydra: Preserving Ensemble Diversity for Model Distillation
• The implicit fairness criterion of unconstrained learning
• Fundamental Limits of Ridge-Regularized Empirical Risk Minimization in High Dimensions
• Moment Multicalibration for Uncertainty Estimation
• Distribution-free binary classification: prediction sets, confidence intervals and calibration
• Sample Complexity of Uniform Convergence for Multicalibration

Cited by in corpus (6)

• Assessing Generalization of SGD via Disagreement
• Early-stopped neural networks are consistent
• Theoretical characterization of uncertainty in high-dimensional linear classification
• Understanding the Under-Coverage Bias in Uncertainty Estimation
• Explaining generalization in deep learning: progress and fundamental limits
• On Deep Neural Network Calibration by Regularization and its Impact on Refinement