1.6k citations · 2.6k across the 33 of their papers we have counts for
36 papers
Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks
Jiayuan Ye, Zhenyu Zhu, Fanghui Liu +2
We analytically investigate how over-parameterization of models in randomized machine learning algorithms impacts the information leakage about their training data. Specifically, w…
Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning
Zebang Shen, Jiayuan Ye, Anmin Kang +2
Repeated parameter sharing in federated learning causes significant information leakage about private data, thus defeating its main purpose: data privacy. Mitigating the risk of th…
Bias Propagation in Federated Learning
Hongyan Chang, Reza Shokri
We show that participating in federated learning can be detrimental to group fairness. In fact, the bias of a few parties against under-represented groups (identified by sensitive…
Leave-one-out Distinguishability in Machine Learning
Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou +1
We introduce an analytical framework to quantify the changes in a machine learning algorithm's output distribution following the inclusion of a few data points in its training set,…
On The Impact of Machine Learning Randomness on Group Fairness
Prakhar Ganesh, Hongyan Chang, Martin Strobel +1
Statistical measures for group fairness in machine learning reflect the gap in performance of algorithms across different groups. These measures, however, exhibit a high variance b…
Smaller Language Models are Better Black-box Machine-Generated Text Detectors
Niloofar Mireshghallah, Justus Mattern, Sicun Gao +2
With the advent of fluent generative language models that can produce convincing utterances very similar to those written by humans, distinguishing whether a piece of text is machi…
Forget Unlearning: Towards True Data-Deletion in Machine Learning
Rishav Chourasia, Neil Shah
Unlearning algorithms aim to remove deleted data's influence from trained models at a cost lower than full retraining. However, prior guarantees of unlearning in literature are fla…
Data Privacy and Trustworthy Machine Learning
Martin Strobel, Reza Shokri
The privacy risks of machine learning models is a major concern when training them on sensitive and personal data. We discuss the tradeoffs between data privacy and the remaining g…
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets
Florian Tramèr, Reza Shokri, Ayrton San Joaquin +4
We introduce a new class of attacks on machine learning models. We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak signifi…
Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)
Jiayuan Ye, Reza Shokri
Prior work on differential privacy analysis of randomized SGD algorithms relies on composition theorems, where the implicit (unrealistic) assumption is that the internal state of t…
Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal +2
The wide adoption and application of Masked language models~(MLMs) on sensitive data (from legal to medical) necessitates a thorough quantitative investigation into their privacy v…
What Does it Mean for a Language Model to Preserve Privacy?
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah +2
Natural language reflects our private lives and identities, making its privacy concerns as broad as those of real life. Language models lack the ability to understand the context a…
Enhanced Membership Inference Attacks against Machine Learning Models
Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda +2
How much does a machine learning algorithm leak about its training data, and why? Membership inference attacks are used as an auditing tool to quantify this leakage. In this paper,…
Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent
Rishav Chourasia, Jiayuan Ye, Reza Shokri
What is the information leakage of an iterative randomized learning algorithm about its training data, when the internal state of the algorithm is \emph{private}? How much is the c…
On the Privacy Risks of Algorithmic Fairness
Hongyan Chang, Reza Shokri
Algorithmic fairness and privacy are essential pillars of trustworthy machine learning. Fair machine learning aims at minimizing discrimination against protected groups by, for exa…
SOTERIA: In Search of Efficient Neural Networks for Private Inference
Anshul Aggarwal, Trevor E. Carlson, Reza Shokri +1
ML-as-a-service is gaining popularity where a cloud server hosts a trained model and offers prediction (inference) service to users. In this setting, our objective is to protect th…
Improving Deep Learning with Differential Privacy using Gradient Encoding and Denoising
Milad Nasr, Reza Shokri, Amir houmansadr
Deep learning models leak significant amounts of information about their training datasets. Previous work has investigated training models with differential privacy (DP) guarantees…
ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning
Sasi Kumar Murakonda, Reza Shokri
When building machine learning models using sensitive data, organizations should ensure that the data processed in such systems is adequately protected. For projects involving mach…
Model Explanations with Differential Privacy
Neel Patel, Reza Shokri, Yair Zick
Black-box machine learning models are used in critical decision-making domains, giving rise to several calls for more algorithmic transparency. The drawback is that model explanati…
On Adversarial Bias and the Robustness of Fair Machine Learning
Hongyan Chang, Ta Duy Nguyen, Sasi Kumar Murakonda +2
Optimizing prediction accuracy can come at the expense of fairness. Towards minimizing discrimination against a group, fair machine learning algorithms strive to equalize the behav…
Epione: Lightweight Contact Tracing with Strong Privacy
Ni Trieu, Kareem Shehata, Prateek Saxena +2
Contact tracing is an essential tool in containing infectious diseases such as COVID-19. Many countries and research groups have launched or announced mobile apps to facilitate con…
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
Hongyan Chang, Virat Shejwalkar, Reza Shokri +1
Collaborative (federated) learning enables multiple parties to train a model without sharing their private data, but through repeated sharing of the parameters of their local model…
Robust Membership Encoding: Inference Attacks and Copyright Protection for Deep Learning
Congzheng Song, Reza Shokri
Machine learning as a service (MLaaS), and algorithm marketplaces are on a rise. Data holders can easily train complex models on their data using third party provided learning code…
On the Privacy Risks of Model Explanations
Reza Shokri, Martin Strobel, Yair Zick
Privacy and transparency are two key foundations of trustworthy machine learning. Model explanations offer insights into a model's decisions on input data, whereas privacy is prima…