Invariant Representations without Adversarial Training
arXiv:1805.09458
Abstract
Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation. Unfortunately, learning representations that exhibit invariance to arbitrary nuisance factors yet remain useful for other tasks is challenging. Existing approaches cast the trade-off between task performance and invariance in an adversarial way, using an iterative minimax optimization. We show that adversarial training is unnecessary and sometimes counter-productive; we instead cast invariant representation learning as a single information-theoretic objective that can be directly optimized. We demonstrate that this approach matches or exceeds performance of state-of-the-art adversarial approaches for learning fair representations and for generative modeling with controllable transformations.
NeurIPS 2018, with corrections
References in corpus (5)
Cited by in corpus (24)
- Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
- DIVA: Domain Invariant Variational Autoencoders
- Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
- Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs
- Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
- Fair Representation: Guaranteeing Approximate Multiple Group Fairness for Unknown Tasks
- Desiderata for Representation Learning: A Causal Perspective
- Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition
- EEG-based Texture Roughness Classification in Active Tactile Exploration with Invariant Representation Learning Networks
- Independent Distribution Regularization for Private Graph Embedding
- Discovery and Separation of Features for Invariant Representation Learning
- Harmonization and the Worst Scanner Syndrome
- Disentangling representations of retinal images with generative models
- An Information-theoretic Approach to Distribution Shifts
- Adversarial Disentanglement with Grouped Observations
- Multi-type Disentanglement without Adversarial Training
- Learning Fair and Interpretable Representations via Linear Orthogonalization
- Inverse Learning of Symmetries
- Robust Classification under Class-Dependent Domain Shift
- Addressing Fairness, Bias and Class Imbalance in Machine Learning: the FBI-loss
- Variation Network: Learning High-level Attributes for Controlled Input Manipulation
- Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptation
- From abstract items to latent spaces to observed data and back: Compositional Variational Auto-Encoder
- Invariant Representations with Stochastically Quantized Neural Networks