Learning Anonymized Representations with Adversarial Neural Networks

Abstract

Statistical methods protecting sensitive information or the identity of the data owner have become critical to ensure privacy of individuals as well as of organizations. This paper investigates anonymization methods based on representation learning and deep neural networks, and motivated by novel information theoretical bounds. We introduce a novel training objective for simultaneously training a predictor over target variables of interest (the regular labels) while preventing an intermediate representation to be predictive of the private labels. The architecture is based on three sub-networks: one going from input to representation, one from representation to predicted regular labels, and one from representation to predicted private labels. The training procedure aims at learning representations that preserve the relevant part of the information (about regular labels) while dismissing information about the private labels which correspond to the identity of a person. We demonstrate the success of this approach for two distinct classification versus anonymization tasks (handwritten digits and sentiment analysis).

20 pages, 6 figures

References in corpus (3)

• Deep Learning with Differential Privacy
• Secure Multiterminal Source Coding with Side Information at the Eavesdropper
• Learning with Differential Privacy: Stability, Learnability and the Sufficiency and Necessity of ERM Principle

Cited by in corpus (12)

• Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?
• Adversarial training approach for local data debiasing
• Training with the Invisibles: Obfuscating Images to Share Safely for Learning Visual Recognition Models
• Defending against Reconstruction Attack in Vertical Federated Learning
• Trade-offs and Guarantees of Adversarial Representation Learning for Information Obfuscation
• Unsupervised Information Obfuscation for Split Inference of Neural Networks
• Generative Adversarial User Privacy in Lossy Single-Server Information Retrieval
• Anonymizing Sensor Data on the Edge: A Representation Learning and Transformation Approach
• G2R Bound: A Generalization Bound for Supervised Learning from GAN-Synthetic Data
• Learning Realistic Patterns from Unrealistic Stimuli: Generalization and Data Anonymization
• Learning Invariant Representations for Sentiment Analysis: The Missing Material is Datasets
• A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification