72 citations · 394 across the 35 of their papers we have counts for
17 papers · 1 filter
Faster Differentially Private Convex Optimization via Second-Order Methods
Arun Ganesh, Mahdi Haghifam, Thomas Steinke +1
Differentially private (stochastic) gradient descent is the workhorse of DP private machine learning in both the convex and non-convex settings. Without privacy constraints, second…
Privacy Auditing with One (1) Training Run
Thomas Steinke, Milad Nasr, Matthew Jagielski
We propose a scheme for auditing differentially private machine learning systems with a single training run. This exploits the parallelism of being able to add or remove multiple t…
Why Is Public Pretraining Necessary for Private Model Training?
Arun Ganesh, Mahdi Haghifam, Milad Nasr +5
In the privacy-utility tradeoff of a model trained on benchmark language and vision tasks, remarkable improvements have been widely reported with the use of pretraining on publicly…
Debugging Differential Privacy: A Case Study for Privacy Auditing
Florian Tramer, Andreas Terzis, Thomas Steinke +3
Differential Privacy can provide provable privacy guarantees for training data in machine learning. However, the presence of proofs does not preclude the presence of errors. Inspir…
Public Data-Assisted Mirror Descent for Private Model Training
Ehsan Amid, Arun Ganesh, Rajiv Mathews +6
In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training. (Here, public d…
Hyperparameter Tuning with Renyi Differential Privacy
Nicolas Papernot, Thomas Steinke
For many differentially private algorithms, such as the prominent noisy stochastic gradient descent (DP-SGD), the analysis needed to bound the privacy leakage of a single training…
PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes
Peter Grünwald, Thomas Steinke, Lydia Zakynthinou
We give a novel, unified derivation of conditional PAC-Bayesian and mutual information (MI) generalization bounds. We derive conditional MI bounds as an instance, with special choi…
Leveraging Public Data for Practical Private Query Release
Terrance Liu, Giuseppe Vietri, Thomas Steinke +2
In many statistical problems, incorporating priors can significantly improve performance. However, the use of prior knowledge in differentially private query release has remained u…
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation
Peter Kairouz, Ziyu Liu, Thomas Steinke
We consider training models on private data that are distributed across user devices. To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy su…
New Oracle-Efficient Algorithms for Private Synthetic Data Release
Giuseppe Vietri, Grace Tian, Mark Bun +2
We present three new algorithms for constructing differentially private synthetic data---a sanitized version of a sensitive dataset that approximately preserves the answers to a la…
Reasoning About Generalization via Conditional Mutual Information
Thomas Steinke, Lydia Zakynthinou
We provide an information-theoretic framework for studying the generalization properties of machine learning algorithms. Our framework ties together existing approaches, including…
A Hybrid Approach to Privacy-Preserving Federated Learning
Stacey Truex, Nathalie Baracaldo, Ali Anwar +4
Federated learning facilitates the collaborative training of models without the sharing of raw data. However, recent attacks demonstrate that simply maintaining data locality durin…
The Limits of Post-Selection Generalization
Kobbi Nissim, Adam Smith, Thomas Steinke +2
While statistics and machine learning offer numerous methods for ensuring generalization, these methods often fail in the presence of adaptivity---the common practice in which the…
Calibrating Noise to Variance in Adaptive Data Analysis
Vitaly Feldman, Thomas Steinke
Datasets are often used multiple times and each successive analysis may depend on the outcome of previous analyses. Standard techniques for ensuring generalization and statistical…
Generalization for Adaptively-chosen Estimators via Stable Median
Vitaly Feldman, Thomas Steinke
Datasets are often reused to perform multiple statistical analyses in an adaptive way, in which each analysis may depend on the outcomes of previous analyses on the same dataset. S…
Algorithmic Stability for Adaptive Data Analysis
Raef Bassily, Kobbi Nissim, Adam Smith +3
Adaptivity is an important feature of data analysis---the choice of questions to ask about a dataset often depends on previous interactions with the same dataset. However, statisti…
More General Queries and Less Generalization Error in Adaptive Data Analysis
Raef Bassily, Adam Smith, Thomas Steinke +1
Adaptivity is an important feature of data analysis---typically the choice of questions asked about a dataset depends on previous interactions with the same dataset. However, gener…