158 citations · 528 across the 70 of their papers we have counts for
73 papers
Differentially Private Mixing of Public Datasets Improves Private Learning
Yufei Chen, Tejumade Afonja, Anvith Thudi +1
Many machine learning applications involve sensitive data and therefore require training under differential privacy (DP). However, DP training often degrades model utility. In some…
Certified in Theory, Broken in Practice: Assumption Gaps in Cryptographic Model Certification
Carter Luck, Olive Franzese-McLaughlin, Elisaweta Masserova +3
Privacy-preserving machine learning auditing protocols allow auditors to assess models for properties such as accuracy or fairness, without revealing their internals or training da…
Gauss-Newton Unlearning for the LLM Era
Lev McKinney, Anvith Thudi, Juhan Bae +4
Standard large language model training can create models that produce outputs their trainer deems unacceptable in deployment. The probability of these outputs can be reduced using…
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
Hanna Foerster, Tom Blanchard, Kristina Nikolić +6
AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guaran…
Efficient Public Verification of Private ML via Regularization
Zoë Ruha Bell, Anvith Thudi, Olive Franzese-McLaughlin +2
Training with differential privacy (DP) guarantees dataset members that they cannot be identified by users of the released model. However, those data providers, and, in general, th…
What Does It Take to Build a Performant Selective Classifier?
Stephan Rabanser, Nicolas Papernot
Selective classifiers improve model reliability by abstaining on inputs the model deems uncertain. However, few practical approaches achieve the gold-standard performance of a perf…
Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning
Roy Rinberg, Ilia Shumailov, Vikrant Singhal +2
Differential privacy (DP) is obtained by randomizing a data analysis algorithm, which necessarily introduces a tradeoff between its utility and privacy. Many DP mechanisms are buil…
Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention
Stephan Rabanser, Ali Shahin Shamsabadi, Olive Franzese +3
Cautious predictions -- where a machine learning model abstains when uncertain -- are crucial for limiting harmful errors in safety-critical applications. In this work, we identify…
Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings
Angéline Pouget, Mohammad Yaghini, Stephan Rabanser +1
Deploying machine learning models in safety-critical domains poses a key challenge: ensuring reliable model performance on downstream user data without access to ground truth label…
Leveraging Per-Instance Privacy for Machine Unlearning
Nazanin Mohammadi Sepahvand, Anvith Thudi, Berivan Isik +5
We present a principled, per-instance approach to quantifying the difficulty of unlearning via fine-tuning. We begin by sharpening an analysis of noisy gradient descent for unlearn…
Private Rate-Constrained Optimization with Applications to Fair Learning
Mohammad Yaghini, Tudor Cebere, Michael Menart +2
Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds,…
Language Models May Verbatim Complete Text They Were Not Explicitly Trained On
Ken Ziyu Liu, Christopher A. Choquette-Choo, Matthew Jagielski +4
An important question today is whether a given text was used to train a large language model (LLM). A \emph{completion} test is often employed: check if the LLM completes a suffici…
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research
A. Feder Cooper, Christopher A. Choquette-Choo, Miranda Bogen +34
"Machine unlearning" is a popular proposed solution for mitigating the existence of content in an AI model that is problematic for legal or moral reasons, including privacy, copyri…
On the Privacy Risk of In-context Learning
Haonan Duan, Adam Dziedzic, Mohammad Yaghini +2
Large language models (LLMs) are excellent few-shot learners. They can perform a wide variety of tasks purely based on natural language prompts provided to them. These prompts cont…
Breach By A Thousand Leaks: Unsafe Information Leakage in `Safe' AI Responses
David Glukhov, Ziwen Han, Ilia Shumailov +2
Vulnerability of Frontier language models to misuse and jailbreaks has prompted the development of safety measures like filters and alignment training in an effort to ensure safety…
UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
Ilia Shumailov, Jamie Hayes, Eleni Triantafillou +6
Exact unlearning was first introduced as a privacy mechanism that allowed a user to retract their data from machine learning models on request. Shortly after, inexact schemes were…
LLM Dataset Inference: Did you train on my dataset?
Pratyush Maini, Hengrui Jia, Nicolas Papernot +1
The proliferation of large language models (LLMs) in the real world has come with a rise in copyright cases against companies for training their models on unlicensed data from the…
Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model
Tudor Cebere, Aurélien Bellet, Nicolas Papernot
Machine learning models can be trained with formal privacy guarantees via differentially private optimizers such as DP-SGD. In this work, we focus on a threat model where the adver…
Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias
Sierra Wyllie, Ilia Shumailov, Nicolas Papernot
Model-induced distribution shifts (MIDS) occur as previous model outputs pollute new model training sets over generations of models. This is known as model collapse in the case of…
Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
Jamie Hayes, Ilia Shumailov, Eleni Triantafillou +2
The high cost of model training makes it increasingly desirable to develop techniques for unlearning. These techniques seek to remove the influence of a training example without ha…
Architectural Neural Backdoors from First Principles
Harry Langford, Ilia Shumailov, Yiren Zhao +2
While previous research backdoored neural networks by changing their parameters, recent work uncovered a more insidious threat: backdoors embedded within the definition of the netw…
Regulation Games for Trustworthy Machine Learning
Mohammad Yaghini, Patty Liu, Franziska Boenisch +1
Existing work on trustworthy machine learning (ML) often concentrates on individual aspects of trust, such as fairness or privacy. Additionally, many techniques overlook the distin…
Fast Exact Unlearning for In-Context Learning Data for LLMs
Andrei I. Muresanu, Anvith Thudi, Michael R. Zhang +1
Modern machine learning models are expensive to train, and there is a growing concern about the challenge of retroactively removing specific training data. Achieving exact unlearni…
Decentralised, Collaborative, and Privacy-preserving Machine Learning for Multi-Hospital Data
Congyu Fang, Adam Dziedzic, Lin Zhang +5
Machine Learning (ML) has demonstrated its great potential on medical data analysis. Large datasets collected from diverse sources and settings are essential for ML models in healt…