5 papers
Reliable AI-Assisted Decisions via Selective Auditing
Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani +1
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimati…
Online Quantile Omniprediction for Proper Losses
Hyunsuk Kim, Isaac Gibbs, Michael W. Mahoney +1
The task of constructing a predictor that is simultaneously accurate under multiple evaluation metrics has been popularized recently under the name omniprediction. Prior work in th…
Locally Adaptive Multi-Objective Learning
Jivat Neet Kaur, Isaac Gibbs, Michael I. Jordan
We consider the general problem of learning a predictor that satisfies multiple objectives of interest simultaneously, a broad framework that captures a range of specific learning…
Calibrated Multi-Level Quantile Forecasting
Tiffany Ding, Isaac Gibbs, Ryan J. Tibshirani
We develop an online method that guarantees calibration of quantile forecasts at multiple quantile levels simultaneously. In this work, a sequence of quantile forecasts is said to…
Sample-Efficient Omniprediction for Proper Losses
Isaac Gibbs, Ryan J. Tibshirani
We consider the problem of constructing probabilistic predictions that lead to accurate decisions when employed by downstream users to inform actions. For a single decision maker,…