Conformal Prediction Under Covariate Shift

Abstract

We extend conformal prediction methodology beyond the case of exchangeable data. In particular, we show that a weighted version of conformal prediction can be used to compute distribution-free prediction intervals for problems in which the test and training covariate distributions differ, but the likelihood ratio between these two distributions is known---or, in practice, can be estimated accurately with access to a large set of unlabeled data (test covariate points). Our weighted extension of conformal prediction also applies more generally, to settings in which the data satisfies a certain weighted notion of exchangeability. We discuss other potential applications of our new conformal methodology, including latent variable and missing data problems.

arXiv comment17 pages, 4 figures

References in corpus (1)

• A tutorial on conformal prediction

Cited by in corpus (11)

• Robust Validation: Confident Predictions Even When Distributions Shift
• Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design
• A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
• Valid model-free spatial prediction
• Is novelty predictable?
• What is a randomization test?
• A Two-Sample Conditional Distribution Test Using Conformal Prediction and Weighted Rank Sum
• Private Prediction Sets
• Model-free Bootstrap and Conformal Prediction in Regression: Conditionality, Conjecture Testing, and Pertinent Prediction Intervals
• Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data
• Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketching