6 papers
Truthful Calibration Measures for Sequential Prediction
Anagha Gokul, Jason Hartline, Lunjia Hu +2
Calibration requires probabilistic reports to be conditionally unbiased and reliably interpretable as probabilities. A calibration measure assigns numerical error to miscalibrated…
Testable and Actionable Calibration for Full Swap Regret
Konstantina Bairaktari, Lunjia Hu, Huy L. Nguyen +1
AI generated predictions increasingly inform decision making in critical tasks, and therefore must be trustworthy. One widely used measure of trustworthiness is calibration, which…
Near-optimal Swap Regret Minimization for Convex Losses
Lunjia Hu, Jon Schneider, Yifan Wu
We give a randomized online algorithm that guarantees near-optimal expected swap regret against any sequence of adaptively chosen Lipschitz convex losse…
Truthful Calibration Errors for Multi-Class Prediction
Yuxuan Lu, Yifan Wu, Jason Hartline +1
Calibrated predictions are useful because their numerical values can be interpreted as probabilities. Calibration errors are therefore widely used to evaluate, compare, and tune pr…
Calibration through the Lens of Indistinguishability
Parikshit Gopalan, Lunjia Hu
Calibration is a classical notion from the forecasting literature which aims to address the question: how should predicted probabilities be interpreted? In a world where we only ge…
A Perfectly Truthful Calibration Measure
Jason Hartline, Lunjia Hu, Yifan Wu
Calibration requires that predictions are conditionally unbiased and, therefore, reliably interpretable as probabilities. A calibration measure quantifies how far a predictor is fr…