Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data
arXiv:2403.19950 · doi:10.1609/aaai.v38i15.29673
Abstract
Out-of-distribution (OOD) generalization has attracted increasing research attention in recent years, due to its promising experimental results in real-world applications. In this paper,we study the confidence set prediction problem in the OOD generalization setting. Split conformal prediction (SCP) is an efficient framework for handling the confidence set prediction problem. However, the validity of SCP requires the examples to be exchangeable, which is violated in the OOD setting. Empirically, we show that trivially applying SCP results in a failure to maintain the marginal coverage when the unseen target domain is different from the source domain. To address this issue, we develop a method for forming confident prediction sets in the OOD setting and theoretically prove the validity of our method. Finally, we conduct experiments on simulated data to empirically verify the correctness of our theory and the validity of our proposed method.
References in corpus (17)
- Explaining and Harnessing Adversarial Examples
- Intriguing properties of neural networks
- Towards Deep Learning Models Resistant to Adversarial Attacks
- A tutorial on conformal prediction
- Deep CORAL: Correlation Alignment for Deep Domain Adaptation
- To Trust Or Not To Trust A Classifier
- Towards Out-Of-Distribution Generalization: A Survey
- Learning to Generalize: Meta-Learning for Domain Generalization
- Minimax Statistical Learning with Wasserstein Distances
- Conformal Prediction Under Covariate Shift
- Understanding the Failure Modes of Out-of-Distribution Generalization
- Robust Validation: Confident Predictions Even When Distributions Shift
- VC Classes are Adversarially Robustly Learnable, but Only Improperly
- Uncertainty Sets for Image Classifiers using Conformal Prediction
- Adaptive Conformal Inference Under Distribution Shift
- Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
- Split Conformal Prediction and Non-Exchangeable Data