Private Prediction Sets

Abstract

In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that provide uncertainty quantification -- they provably cover the true response with a user-specified probability, such as 90%. One might hope that when used with privately-trained models, conformal prediction would yield privacy guarantees for the resulting prediction sets; unfortunately, this is not the case. To remedy this key problem, we develop a method that takes any pre-trained predictive model and outputs differentially private prediction sets. Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine. This subroutine compensates for the noise introduced to preserve privacy in order to guarantee correct coverage. We evaluate the method on large-scale computer vision datasets.

arXiv commentCode available at https://github.com/aangelopoulos/private_prediction_sets

References in corpus (28)

• Deep Learning with Differential Privacy
• RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
• A tutorial on conformal prediction
• Differentially Private Empirical Risk Minimization
• Prochlo: Strong Privacy for Analytics in the Crowd
• Collecting Telemetry Data Privately
• Least Ambiguous Set-Valued Classifiers with Bounded Error Levels
• Conformal Prediction Under Covariate Shift
• Robust Validation: Confident Predictions Even When Distributions Shift
• Distribution-Free Predictive Inference For Regression
• Conformal Inference of Counterfactuals and Individual Treatment Effects
• Uncertainty Sets for Image Classifiers using Conformal Prediction
• Cautious Deep Learning
• A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
• Flexible distribution-free conditional predictive bands using density estimators
• Classification with Valid and Adaptive Coverage
• Conformal prediction with localization
• Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction
• PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction
• Locally Private Mean Estimation: Z-test and Tight Confidence Intervals
• Moment Multicalibration for Uncertainty Estimation
• Conformal calibrators
• Prediction and outlier detection in classification problems
• Oracle Efficient Private Non-Convex Optimization
• Make Up Your Mind: The Price of Online Queries in Differential Privacy
• Generalization for Adaptively-chosen Estimators via Stable Median
• Online Multivalid Learning: Means, Moments, and Prediction Intervals
• Differentially Private Confidence Intervals for Empirical Risk Minimization

Cited by in corpus (2)

• Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
• Conformal Prediction for Offensive Security