8 citations · 11 across the 21 of their papers we have counts for
26 papers
Compound p-values for competition-based testing
Junu Lee, Rina Foygel Barber, Richard J. Samworth +1
Standard competition-based multiple testing methods -- such as the knockoff filter and the target-decoy procedure -- are typically formulated as head-to-head competitions between a…
A Ranking Approach for Measuring Calibration
Anirban Chatterjee, Rina Foygel Barber
When providing forecasted probabilities with a predictive model, the ideal model offers perfect calibration: the true probability of the outcome (i.e., the probability that )…
Algorithmic stability via ensembling
Rina Foygel Barber, Richard J. Samworth
Algorithmic stability refers to the property of an algorithm being insensitive to perturbations of the input data, where the type of perturbation may vary depending on the setting.…
An analysis of binary isotonic regression: degrees of freedom and implications for calibration
Raphael Rossellini, Rina Foygel Barber, Zhimei Ren +1
Isotonic regression is a canonical tool for estimating monotone functions and calibrating probabilistic predictors. We provide a fully sharp finite-sample characterization of its w…
Local permutation tests for conditional independence: an adaptive binning perspective
David Chen, Rohan Hore, Rina Foygel Barber
In this work, we study the problem of testing conditional independence between random variables and given a confounder . The local permutation test (LPT) offers a princi…
Approximating full conformal prediction: distribution free guarantees via the tournament correction
Aabesh Bhattacharyya, Boxuan Zhang, Rina Foygel Barber
Conformal prediction is a framework for providing prediction intervals with distribution-free validity, guaranteeing predictive coverage for data drawn from any distribution. Its t…
Conformal Prediction with Macro-Coverage Guarantees
Aabesh Bhattacharyya, Tiffany Ding, Rina Foygel Barber
Prediction sets should have high coverage to be useful, but some coverage notions are more practically relevant than others. In the classification setting, class-conditional covera…
Distribution-free two-sample testing with blurred total variation distance
Rohan Hore, Rina Foygel Barber
Two-sample testing, where we aim to determine whether two distributions are equal or not equal based on samples from each one, is challenging if we cannot place assumptions on the…
False positive control in time series coincidence detection
Ruiting Liang, Samuel Dyson, Rina Foygel Barber +1
We study the problem of coincidence detection in time series data, where we aim to determine whether the appearance of simultaneous or near-simultaneous events in two time series i…
Stability and Accuracy Trade-offs in Statistical Estimation
Abhinav Chakraborty, Yuetian Luo, Rina Foygel Barber
Algorithmic stability is a central concept in statistics and learning theory that measures how sensitive an algorithm's output is to small changes in the training data. Stability p…
Conditioning on posterior samples for flexible frequentist goodness-of-fit testing
Ritwik Bhaduri, Aabesh Bhattacharyya, Rina Foygel Barber +1
Tests of goodness of fit are used in nearly every domain where statistics is applied. One powerful and flexible approach is to sample artificial data sets that are exchangeable wit…
False discovery rate control with compound p-values
Rina Foygel Barber, Richard J Samworth
In the setting of multiple testing, compound p-values generalize p-values by asking for superuniformity to hold only \emph{on average} across all true nulls. We study the propertie…
Mosaic inference on panel data
Asher Spector, Rina Foygel Barber, Emmanuel Candès
Analysis of panel data via linear regression is widespread across disciplines. To perform statistical inference, such analyses typically assume that clusters of observations are jo…
Assumption-free stability for ranking problems
Ruiting Liang, Jake A. Soloff, Rina Foygel Barber +1
In this work, we consider ranking problems among a finite set of candidates: for instance, selecting the top- items among a larger list of candidates or obtaining the full ranki…
Can a calibration metric be both testable and actionable?
Raphael Rossellini, Jake A. Soloff, Rina Foygel Barber +2
Forecast probabilities often serve as critical inputs for binary decision making. In such settings, calibration$\unicode{x2014}$ensuring forecasted probabilities match empirical fr…
Are all models wrong? Fundamental limits in distribution-free empirical model falsification
Manuel M. Müller, Yuetian Luo, Rina Foygel Barber
In statistics and machine learning, when we train a fitted model on available data, we typically want to ensure that we are searching within a model class that contains at least on…
Testing conditional independence under isotonicity
Rohan Hore, Jake A. Soloff, Rina Foygel Barber +1
We propose a test of the conditional independence of random variables and~ given~ under the additional assumption that is stochastically nondecreasing in~. The wel…
Theoretical Foundations of Conformal Prediction
Anastasios N. Angelopoulos, Rina Foygel Barber, Stephen Bates
This book is about conformal prediction and related inferential techniques that build on permutation tests and exchangeability. These techniques are useful in a diverse array of ta…
Distributionally robust risk evaluation with an isotonic constraint
Yu Gui, Rina Foygel Barber, Cong Ma
Statistical learning under distribution shift is challenging when neither prior knowledge nor fully accessible data from the target distribution is available. Distributionally robu…
Stability via resampling: statistical problems beyond the real line
Jake A. Soloff, Rina Foygel Barber, Rebecca Willett
Model averaging techniques based on resampling methods (such as bootstrapping or subsampling) have been utilized across many areas of statistics, often with the explicit goal of pr…
Is Algorithmic Stability Testable? A Unified Framework under Computational Constraints
Yuetian Luo, Rina Foygel Barber
Algorithmic stability is a central notion in learning theory that quantifies the sensitivity of an algorithm to small changes in the training data. If a learning algorithm satisfie…
Building a stable classifier with the inflated argmax
Jake A. Soloff, Rina Foygel Barber, Rebecca Willett
We propose a new framework for algorithmic stability in the context of multiclass classification. In practice, classification algorithms often operate by first assigning a continuo…
The mosaic permutation test: an exact and nonparametric goodness-of-fit test for factor models
Asher Spector, Rina Foygel Barber, Trevor Hastie +2
Financial firms often rely on fundamental factor models to explain correlations among asset returns and manage risk. Yet after major events, e.g., COVID-19, analysts may reassess w…
The Limits of Assumption-free Tests for Algorithm Performance
Yuetian Luo, Rina Foygel Barber
Algorithm evaluation and comparison are fundamental questions in machine learning and statistics -- how well does an algorithm perform at a given modeling task, and which algorithm…