Controllable Invariance through Adversarial Feature Learning

Abstract

Learning meaningful representations that maintain the content necessary for a particular task while filtering away detrimental variations is a problem of great interest in machine learning. In this paper, we tackle the problem of learning representations invariant to a specific factor or trait of data. The representation learning process is formulated as an adversarial minimax game. We analyze the optimal equilibrium of such a game and find that it amounts to maximizing the uncertainty of inferring the detrimental factor given the representation while maximizing the certainty of making task-specific predictions. On three benchmark tasks, namely fair and bias-free classification, language-independent generation, and lighting-independent image classification, we show that the proposed framework induces an invariant representation, and leads to better generalization evidenced by the improved performance.

arXiv commentNIPS 2017

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Adversarial Discriminative Domain Adaptation
• Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

Cited by in corpus (19)

• Unsupervised Machine Translation Using Monolingual Corpora Only
• Subject-Aware Contrastive Learning for Biosignals
• Invariant Representations from Adversarially Censored Autoencoders
• Adversarial Removal of Demographic Attributes from Text Data
• GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
• NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature
• Discriminative-Generative Representation Learning for One-Class Anomaly Detection
• Disentangling representations of retinal images with generative models
• Adjusting for Confounding in Unsupervised Latent Representations of Images
• Style Obfuscation by Invariance
• An Information-theoretic Approach to Distribution Shifts
• Clustering-Based Representation Learning through Output Translation and Its Application to Remote--Sensing Images
• Improving Image Clustering through Sample Ranking and Its Application to remote--sensing images
• Event-Related Bias Removal for Real-time Disaster Events
• Projection-wise Disentangling for Fair and Interpretable Representation Learning: Application to 3D Facial Shape Analysis
• A Domain Generalization Perspective on Listwise Context Modeling
• Invariant Representations with Stochastically Quantized Neural Networks
• Learning to Collaborate for User-Controlled Privacy
• Don't Judge Me by My Face : An Indirect Adversarial Approach to Remove Sensitive Information From Multimodal Neural Representation in Asynchronous Job Video Interviews