5 papers
CalArena: A Large-Scale Post-Hoc Calibration Benchmark
Eugène Berta, David Holzmüller, Francis Bach +1
Reliable probability estimates are critical in many machine learning applications, yet modern classifiers are often poorly calibrated. Post-hoc calibration provides a simple and wi…
Multivariate Standardized Residuals for Conformal Prediction
Sacha Braun, Eugène Berta, Michael I. Jordan +1
While split conformal prediction guarantees marginal coverage, approaching the stronger property of conditional coverage is essential for reliable uncertainty quantification. Naive…
Structured Matrix Scaling for Multi-Class Calibration
Eugène Berta, David Holzmüller, Michael I. Jordan +1
Post-hoc recalibration methods are widely used to ensure that classifiers provide faithful probability estimates. We argue that parametric recalibration functions based on logistic…
A Variational Estimator for Calibration Errors
Eugène Berta, Sacha Braun, David Holzmüller +2
Calibration$\unicode{x2014}$the problem of ensuring that predicted probabilities align with observed class frequencies$\unicode{x2014}$is a basic desideratum for reliable predictio…
Rethinking Early Stopping: Refine, Then Calibrate
Eugène Berta, David Holzmüller, Michael I. Jordan +1
Machine learning classifiers often produce probabilistic predictions that are critical for accurate and interpretable decision-making in various domains. The quality of these predi…