6 papers
Toward Calibrated Mixture-of-Experts Under Distribution Shift
Gina Wong, Drew Prinster, Suchi Saria +2
Calibration aligns a model's predictive uncertainty with the frequencies of its empirical outcomes and is important for understanding and trusting reported probabilities. Recent wo…
Conformal Policy Control
Drew Prinster, Clara Fannjiang, Ji Won Park +4
An agent must try new behaviors to explore and improve. In high-stakes environments, an agent that violates safety constraints may cause harm and must be taken offline, curtailing…
Improving Coverage in Combined Prediction Sets with Weighted p-values
Gina Wong, Drew Prinster, Suchi Saria +2
Conformal prediction quantifies the uncertainty of machine learning models by augmenting point predictions with valid prediction sets. For complex scenarios involving multiple tria…
WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales
Drew Prinster, Xing Han, Anqi Liu +1
Responsibly deploying artificial intelligence (AI) / machine learning (ML) systems in high-stakes settings arguably requires not only proof of system reliability, but also continua…
Prescribing Decision Conservativeness in Two-Stage Power Markets: A Distributionally Robust End-to-End Approach
Zhirui Liang, Qi Li, Anqi Liu +1
This paper presents an end-to-end framework for calibrating wind power forecast models to minimize operational costs in two-stage power markets, where the first stage involves a di…
Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)
Drew Prinster, Samuel Stanton, Anqi Liu +1
As artificial intelligence (AI) / machine learning (ML) gain widespread adoption, practitioners are increasingly seeking means to quantify and control the risk these systems incur.…