5 papers
Testable and Actionable Calibration for Full Swap Regret
Konstantina Bairaktari, Lunjia Hu, Huy L. Nguyen +1
AI generated predictions increasingly inform decision making in critical tasks, and therefore must be trustworthy. One widely used measure of trustworthiness is calibration, which…
Truthful Calibration Errors for Multi-Class Prediction
Yuxuan Lu, Yifan Wu, Jason Hartline +1
Calibrated predictions are useful because their numerical values can be interpreted as probabilities. Calibration errors are therefore widely used to evaluate, compare, and tune pr…
A Perfectly Truthful Calibration Measure
Jason Hartline, Lunjia Hu, Yifan Wu
Calibration requires that predictions are conditionally unbiased and, therefore, reliably interpretable as probabilities. A calibration measure quantifies how far a predictor is fr…
Near-optimal Swap Regret Minimization for Convex Losses
Lunjia Hu, Jon Schneider, Yifan Wu
We give a randomized online algorithm that guarantees near-optimal expected swap regret against any sequence of adaptively chosen Lipschitz convex losse…
Calibration through the Lens of Indistinguishability
Parikshit Gopalan, Lunjia Hu
Calibration is a classical notion from the forecasting literature which aims to address the question: how should predicted probabilities be interpreted? In a world where we only ge…