157 citations · 741 across the 64 of their papers we have counts for
14 papers · 1 filter
Differentially Private Image Classification from Features
Harsh Mehta, Walid Krichene, Abhradeep Thakurta +2
Leveraging transfer learning has recently been shown to be an effective strategy for training large models with Differential Privacy (DP). Moreover, somewhat surprisingly, recent w…
Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
Christopher A. Choquette-Choo, H. Brendan McMahan, Keith Rush +1
We introduce new differentially private (DP) mechanisms for gradient-based machine learning (ML) with multiple passes (epochs) over a dataset, substantially improving the achievabl…
Fully Adaptive Composition for Gaussian Differential Privacy
Adam Smith, Abhradeep Thakurta
We show that Gaussian Differential Privacy, a variant of differential privacy tailored to the analysis of Gaussian noise addition, composes gracefully even in the presence of a ful…
Fine-Tuning with Differential Privacy Necessitates an Additional Hyperparameter Search
Yannis Cattan, Christopher A. Choquette-Choo, Nicolas Papernot +1
Models need to be trained with privacy-preserving learning algorithms to prevent leakage of possibly sensitive information contained in their training data. However, canonical algo…
Sample-Efficient Personalization: Modeling User Parameters as Low Rank Plus Sparse Components
Soumyabrata Pal, Prateek Varshney, Prateek Jain +5
Personalization of machine learning (ML) predictions for individual users/domains/enterprises is critical for practical recommendation systems. Standard personalization approaches…
Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpoints
Virat Shejwalkar, Arun Ganesh, Rajiv Mathews +5
In this work, we focus on improving the accuracy-variance trade-off for state-of-the-art differentially private machine learning (DP ML) methods. First, we design a general framewo…
(Nearly) Optimal Private Linear Regression via Adaptive Clipping
Prateek Varshney, Abhradeep Thakurta, Prateek Jain
We study the problem of differentially private linear regression where each data point is sampled from a fixed sub-Gaussian style distribution. We propose and analyze a one-pass mi…
Private Matrix Approximation and Geometry of Unitary Orbits
Oren Mangoubi, Yikai Wu, Satyen Kale +2
Consider the following optimization problem: Given matrices and , maximize where varies over the unitary group . Th…
When Does Differentially Private Learning Not Suffer in High Dimensions?
Xuechen Li, Daogao Liu, Tatsunori Hashimoto +4
Large pretrained models can be privately fine-tuned to achieve performance approaching that of non-private models. A common theme in these results is the surprising observation tha…
Measuring Forgetting of Memorized Training Examples
Matthew Jagielski, Om Thakkar, Florian Tramèr +8
Machine learning models exhibit two seemingly contradictory phenomena: training data memorization, and various forms of forgetting. In memorization, models overfit specific trainin…
Large Scale Transfer Learning for Differentially Private Image Classification
Harsh Mehta, Abhradeep Thakurta, Alexey Kurakin +1
Differential Privacy (DP) provides a formal framework for training machine learning models with individual example level privacy. In the field of deep learning, Differentially Priv…
Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimization
Arun Ganesh, Abhradeep Thakurta, Jalaj Upadhyay
In this paper we provide an algorithmic framework based on Langevin diffusion (LD) and its corresponding discretizations that allow us to simultaneously obtain: i) An algorithm for…
Toward Training at ImageNet Scale with Differential Privacy
Alexey Kurakin, Shuang Song, Steve Chien +3
Differential privacy (DP) is the de facto standard for training machine learning (ML) models, including neural networks, while ensuring the privacy of individual examples in the tr…
Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams
Sergey Denisov, Brendan McMahan, Keith Rush +2
Motivated by recent applications requiring differential privacy over adaptive streams, we investigate the question of optimal instantiations of the matrix mechanism in this setting…