Unlearnable Examples: Making Personal Data Unexploitable

Abstract

The volume of "free" data on the internet has been key to the current success of deep learning. However, it also raises privacy concerns about the unauthorized exploitation of personal data for training commercial models. It is thus crucial to develop methods to prevent unauthorized data exploitation. This paper raises the question: \emph{can data be made unlearnable for deep learning models?} We present a type of \emph{error-minimizing} noise that can indeed make training examples unlearnable. Error-minimizing noise is intentionally generated to reduce the error of one or more of the training example(s) close to zero, which can trick the model into believing there is "nothing" to learn from these example(s). The noise is restricted to be imperceptible to human eyes, and thus does not affect normal data utility. We empirically verify the effectiveness of error-minimizing noise in both sample-wise and class-wise forms. We also demonstrate its flexibility under extensive experimental settings and practicability in a case study of face recognition. Our work establishes an important first step towards making personal data unexploitable to deep learning models.

ICLR2021, In International Conference on Learning Representations

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning with Differential Privacy
• Improved Regularization of Convolutional Neural Networks with Cutout
• Learning Face Representation from Scratch
• Understanding Black-box Predictions via Influence Functions
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Theoretically Principled Trade-off between Robustness and Accuracy
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
• ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
• On the Convergence and Robustness of Adversarial Training
• Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
• Generative Poisoning Attack Method Against Neural Networks
• Large image datasets: A pyrrhic win for computer vision?
• A Unified Approach to Interpreting and Boosting Adversarial Transferability

Cited by in corpus (7)

• Fast Yet Effective Machine Unlearning
• Adversarial Examples Make Strong Poisons
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• Can Pretrained Language Models Derive Correct Semantics from Corrupt Subwords under Noise?
• Make Text Unlearnable: Exploiting Effective Patterns to Protect Personal Data
• Disrupting Model Training with Adversarial Shortcuts
• Amicable Aid: Perturbing Images to Improve Classification Performance