Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs
arXiv:2105.12049 · doi:10.1145/3460120.3484533
Abstract
It is known that deep neural networks, trained for the classification of non-sensitive target attributes, can reveal sensitive attributes of their input data through internal representations extracted by the classifier. We take a step forward and show that deep classifiers can be trained to secretly encode a sensitive attribute of their input data into the classifier's outputs for the target attribute, at inference time. Our proposed attack works even if users have a full white-box view of the classifier, can keep all internal representations hidden, and only release the classifier's estimations for the target attribute. We introduce an information-theoretical formulation for such attacks and present efficient empirical implementations for training honest-but-curious (HBC) classifiers: classifiers that can be accurate in predicting their target attribute, but can also exploit their outputs to secretly encode a sensitive attribute. Our work highlights a vulnerability that can be exploited by malicious machine learning service providers to attack their user's privacy in several seemingly safe scenarios; such as encrypted inferences, computations at the edge, or private knowledge distillation. Experimental results on several attributes in two face-image datasets show that a semi-trusted server can train classifiers that are not only perfectly honest but also accurately curious. We conclude by showing the difficulties in distinguishing between standard and HBC classifiers, discussing challenges in defending against this vulnerability of deep classifiers, and enumerating related open directions for future studies.
arXiv commentIn Proceedings of the 2021 ACMSIGSAC Conference on Computer and Communications Security (CCS '21)
References in corpus (26)
- Deep Learning in Neural Networks: An Overview
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Distilling the Knowledge in a Neural Network
- A Unified Approach to Interpreting Model Predictions
- Deep Learning with Differential Privacy
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- The information bottleneck method
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Opening the Black Box of Deep Neural Networks via Information
- Poisoning Attacks against Support Vector Machines
- Training Deep Neural Networks on Noisy Labels with Bootstrapping
- DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environments
- Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
- A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analytics
- On Variational Bounds of Mutual Information
- Invariant Representations without Adversarial Training
- Privacy in Deep Learning: A Survey
- Overlearning Reveals Sensitive Attributes
- Overparameterized Neural Networks Implement Associative Memory
- ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
- Dopamine: Differentially Private Federated Learning on Medical Data
- An Overview of Privacy in Machine Learning
- Layer-wise Characterization of Latent Information Leakage in Federated Learning
- Generalization in multitask deep neural classifiers: a statistical physics approach
- Disentangling Influence: Using Disentangled Representations to Audit Model Predictions