Private Prediction Sets

Abstract

In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that provide uncertainty quantification -- they provably cover the true response with a user-specified probability, such as 90%. One might hope that when used with privately-trained models, conformal prediction would yield privacy guarantees for the resulting prediction sets; unfortunately, this is not the case. To remedy this key problem, we develop a method that takes any pre-trained predictive model and outputs differentially private prediction sets. Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine. This subroutine compensates for the noise introduced to preserve privacy in order to guarantee correct coverage. We evaluate the method on large-scale computer vision datasets.

Code available at https://github.com/aangelopoulos/private_prediction_sets

References in corpus (15)

• RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
• Collecting Telemetry Data Privately
• Robust Validation: Confident Predictions Even When Distributions Shift
• Conformal Inference of Counterfactuals and Individual Treatment Effects
• Uncertainty Sets for Image Classifiers using Conformal Prediction
• A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
• Flexible distribution-free conditional predictive bands using density estimators
• Classification with Valid and Adaptive Coverage
• Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction
• PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction
• Moment Multicalibration for Uncertainty Estimation
• Conformal calibrators
• Prediction and outlier detection in classification problems
• Generalization for Adaptively-chosen Estimators via Stable Median
• Online Multivalid Learning: Means, Moments, and Prediction Intervals

Cited by in corpus (2)

• Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
• Conformal Prediction for Offensive Security