10 papers · 1 filter
A Ranking Approach for Measuring Calibration
Anirban Chatterjee, Rina Foygel Barber
When providing forecasted probabilities with a predictive model, the ideal model offers perfect calibration: the true probability of the outcome (i.e., the probability that )…
Local permutation tests for conditional independence: an adaptive binning perspective
David Chen, Rohan Hore, Rina Foygel Barber
In this work, we study the problem of testing conditional independence between random variables and given a confounder . The local permutation test (LPT) offers a princi…
Approximating full conformal prediction: distribution free guarantees via the tournament correction
Aabesh Bhattacharyya, Boxuan Zhang, Rina Foygel Barber
Conformal prediction is a framework for providing prediction intervals with distribution-free validity, guaranteeing predictive coverage for data drawn from any distribution. Its t…
Conformal Prediction with Macro-Coverage Guarantees
Aabesh Bhattacharyya, Tiffany Ding, Rina Foygel Barber
Prediction sets should have high coverage to be useful, but some coverage notions are more practically relevant than others. In the classification setting, class-conditional covera…
Conditioning on posterior samples for flexible frequentist goodness-of-fit testing
Ritwik Bhaduri, Aabesh Bhattacharyya, Rina Foygel Barber +1
Tests of goodness of fit are used in nearly every domain where statistics is applied. One powerful and flexible approach is to sample artificial data sets that are exchangeable wit…
Mosaic inference on panel data
Asher Spector, Rina Foygel Barber, Emmanuel Candès
Analysis of panel data via linear regression is widespread across disciplines. To perform statistical inference, such analyses typically assume that clusters of observations are jo…