NeuGuard: Lightweight Neuron-Guided Defense against Membership Inference Attacks

Abstract

Membership inference attacks (MIAs) against machine learning models can lead to serious privacy risks for the training dataset used in the model training. In this paper, we propose a novel and effective Neuron-Guided Defense method named NeuGuard against membership inference attacks (MIAs). We identify a key weakness in existing defense mechanisms against MIAs wherein they cannot simultaneously defend against two commonly used neural network based MIAs, indicating that these two attacks should be separately evaluated to assure the defense effectiveness. We propose NeuGuard, a new defense approach that jointly controls the output and inner neurons' activation with the object to guide the model output of training set and testing set to have close distributions. NeuGuard consists of class-wise variance minimization targeting restricting the final output neurons and layer-wise balanced output control aiming to constrain the inner neurons in each layer. We evaluate NeuGuard and compare it with state-of-the-art defenses against two neural network based MIAs, five strongest metric based MIAs including the newly proposed label-only MIA on three benchmark datasets. Results show that NeuGuard outperforms the state-of-the-art defenses by offering much improved utility-privacy trade-off, generality, and overhead.

References in corpus (15)

• Distilling the Knowledge in a Neural Network
• Deep Learning with Differential Privacy
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
• Differentially Private Model Publishing for Deep Learning
• Privacy Risks of Securing Machine Learning Models against Adversarial Examples
• Practical Blind Membership Inference Attack via Differential Comparisons
• Systematic Evaluation of Privacy Risks of Machine Learning Models
• Towards Demystifying Membership Inference Attacks
• Accelerating Data Loading in Deep Neural Network Training
• MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples
• Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture
• Knowledge Cross-Distillation for Membership Privacy
• Membership Inference Attacks and Defenses in Neural Network Pruning