Censoring Representations with an Adversary

Abstract

In practice, there are often explicit constraints on what representations or decisions are acceptable in an application of machine learning. For example it may be a legal requirement that a decision must not favour a particular group. Alternatively it can be that that representation of data must not have identifying information. We address these two related issues by learning flexible representations that minimize the capability of an adversarial critic. This adversary is trying to predict the relevant sensitive variable from the representation, and so minimizing the performance of the adversary ensures there is little or no information in the representation about the sensitive variable. We demonstrate this adversarial approach on two problems: making decisions free from discrimination and removing private information from images. We formulate the adversarial model as a minimax problem, and optimize that minimax objective using a stochastic gradient alternate min-max optimizer. We demonstrate the ability to provide discriminant free representations for standard test problems, and compare with previous state of the art methods for fairness, showing statistically significant improvement across most cases. The flexibility of this method is shown via a novel problem: removing annotations from images, from unaligned training examples of annotated and unannotated images, and with no a priori knowledge of the form of annotation provided to the model.

Paper accepted to ICLR

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Theano: new features and speed improvements
• On distinguishability criteria for estimating generative models
• ScreenAvoider: Protecting Computer Screens from Ubiquitous Cameras
• Generative Class-conditional Autoencoders

Cited by in corpus (57)

• NIPS 2016 Tutorial: Generative Adversarial Networks
• Fairness in Machine Learning: A Survey
• The Measure and Mismeasure of Fairness
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Connecting User and Item Perspectives in Popularity Debiasing for Collaborative Recommendation
• Learning to Protect Communications with Adversarial Neural Cryptography
• Identifying and Correcting Label Bias in Machine Learning
• Measuring and Reducing Gendered Correlations in Pre-trained Models
• Causal Reasoning for Algorithmic Fairness
• An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges
• Training individually fair ML models with Sensitive Subspace Robustness
• FairGAN: Fairness-aware Generative Adversarial Networks
• Generative Adversarial Privacy
• Privacy Adversarial Network: Representation Learning for Mobile Data Privacy
• Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability
• Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
• Inherent Tradeoffs in Learning Fair Representations
• Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
• Algorithmic Decision Making with Conditional Fairness
• A Distributionally Robust Approach to Fair Classification
• Adversarial training approach for local data debiasing
• Conditional Learning of Fair Representations
• Fair Generative Modeling via Weak Supervision
• Rényi Fair Inference
• Invariant Representations from Adversarially Censored Autoencoders
• Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations
• Modeling Techniques for Machine Learning Fairness: A Survey
• Adversarial Removal of Demographic Attributes from Text Data
• Fundamental Limits and Tradeoffs in Invariant Representation Learning
• Interpretable and Fair Boolean Rule Sets via Column Generation
• Ethical Adversaries: Towards Mitigating Unfairness with Adversarial Machine Learning
• Multi-View Data Generation Without View Supervision
• On the Fairness of Disentangled Representations
• Application-driven Privacy-preserving Data Publishing with Correlated Attributes
• K-Beam Minimax: Efficient Optimization for Deep Adversarial Learning
• State of the Art in Fair ML: From Moral Philosophy and Legislation to Fair Classifiers
• Disentanglement for Discriminative Visual Recognition
• VAE/WGAN-Based Image Representation Learning For Pose-Preserving Seamless Identity Replacement In Facial Images
• Distributed generation of privacy preserving data with user customization
• A Cyclically-Trained Adversarial Network for Invariant Representation Learning
• Conditional t-SNE: Complementary t-SNE embeddings through factoring out prior information
• Abstracting Fairness: Oracles, Metrics, and Interpretability
• Towards Reducing Bias in Gender Classification
• Learning Smooth and Fair Representations
• Online Monotone Optimization
• Fair Decision Rules for Binary Classification
• Adversarial Stacked Auto-Encoders for Fair Representation Learning
• Metric-Free Individual Fairness with Cooperative Contextual Bandits
• Fairness-aware Summarization for Justified Decision-Making
• DYSAN: Dynamically sanitizing motion sensor data against sensitive inferences through adversarial networks
• Universal Physiological Representation Learning with Soft-Disentangled Rateless Autoencoders
• Reducing Overlearning through Disentangled Representations by Suppressing Unknown Tasks
• Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers
• Disentangled Adversarial Transfer Learning for Physiological Biosignals
• Transfer Learning in Brain-Computer Interfaces with Adversarial Variational Autoencoders
• Deep Clustering based Fair Outlier Detection