6 papers
Monte Carlo testing: non-asymptotic guarantees without joint exchangeability
Rina Foygel Barber, Aaditya Ramdas
In hypothesis testing, Monte Carlo tests are usually justified either by exact null simulation or by joint exchangeability of the observed data and its simulated copies. This leave…
Conformal prediction after data-dependent model selection
Ruiting Liang, Wanrong Zhu, Rina Foygel Barber
Given a family of pretrained models and a hold-out set, how can we construct a valid conformal prediction set while selecting a model that minimizes the width of the set? If we use…
Approximate co-sufficient sampling with regularization
Wanrong Zhu, Rina Foygel Barber
In this work, we consider the problem of goodness-of-fit (GoF) testing for parametric models. This testing problem involves a composite null hypothesis, due to the unknown values o…
Distribution-free inference with hierarchical data
Yonghoon Lee, Rina Foygel Barber, Rebecca Willett
This paper studies distribution-free inference in settings where the data set has a hierarchical structure -- for example, groups of observations, or repeated measurements. In such…
Algorithmic stability implies training-conditional coverage for distribution-free prediction methods
Ruiting Liang, Rina Foygel Barber
In a supervised learning problem, given a predicted value that is the output of some trained model, how can we quantify our uncertainty around this prediction? Distribution-free pr…
Group-Weighted Conformal Prediction
Aabesh Bhattacharyya, Rina Foygel Barber
Conformal prediction (CP) is a method for constructing a prediction interval around the output of a fitted model, whose validity does not rely on the model being correct--the CP in…