109 citations · 211 across the 11 of their papers we have counts for
11 papers
Can Neural Network Memorization Be Localized?
Pratyush Maini, Michael C. Mozer, Hanie Sedghi +3
Recent efforts at explaining the interplay of memorization and generalization in deep overparametrized networks have posited that neural networks "hard" example…
On User-Level Private Convex Optimization
Badih Ghazi, Pritish Kamath, Ravi Kumar +3
We introduce a new mechanism for stochastic convex optimization (SCO) with user-level differential privacy guarantees. The convergence rates of this mechanism are similar to those…
Regression with Label Differential Privacy
Badih Ghazi, Pritish Kamath, Ravi Kumar +4
We study the task of training regression models with the guarantee of label differential privacy (DP). Based on a global prior distribution on label values, which could be obtained…
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Daphne Ippolito, Florian Tramèr, Milad Nasr +5
Studying data memorization in neural language models helps us understand the risks (e.g., to privacy or copyright) associated with models regurgitating training data and aids in th…
Measuring Forgetting of Memorized Training Examples
Matthew Jagielski, Om Thakkar, Florian Tramèr +8
Machine learning models exhibit two seemingly contradictory phenomena: training data memorization, and various forms of forgetting. In memorization, models overfit specific trainin…
The Privacy Onion Effect: Memorization is Relative
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang +3
Machine learning models trained on private datasets have been shown to leak their private data. While recent work has found that the average data point is rarely leaked, the outlie…
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models
Weiyan Shi, Ryan Shea, Si Chen +3
Protecting large language models from privacy leakage is becoming increasingly crucial with their wide adoption in real-world products. Yet applying differential privacy (DP), a ca…
Counterfactual Memorization in Neural Language Models
Chiyuan Zhang, Daphne Ippolito, Katherine Lee +3
Modern neural language models that are widely used in various NLP tasks risk memorizing sensitive information from their training data. Understanding this memorization is important…
Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation
Yao Qin, Chiyuan Zhang, Ting Chen +3
We investigate the robustness of vision transformers (ViTs) through the lens of their special patch-based architectural structure, i.e., they process an image as a sequence of imag…
Do Vision Transformers See Like Convolutional Neural Networks?
Maithra Raghu, Thomas Unterthiner, Simon Kornblith +2
Convolutional neural networks (CNNs) have so far been the de-facto model for visual data. Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or eve…
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization
Chiyuan Zhang, Maithra Raghu, Jon Kleinberg +1
Central to the success of artificial neural networks is their ability to generalize. But does neural network generalization primarily rely on seeing highly similar training example…