Latent Adversarial Debiasing: Mitigating Collider Bias in Deep Neural Networks
arXiv:2011.11486
Abstract
Collider bias is a harmful form of sample selection bias that neural networks are ill-equipped to handle. This bias manifests itself when the underlying causal signal is strongly correlated with other confounding signals due to the training data collection procedure. In the situation where the confounding signal is easy-to-learn, deep neural networks will latch onto this and the resulting model will generalise poorly to in-the-wild test scenarios. We argue herein that the cause of failure is a combination of the deep structure of neural networks and the greedy gradient-driven learning process used - one that prefers easy-to-compute signals when available. We show it is possible to mitigate against this by generating bias-decoupled training data using latent adversarial debiasing (LAD), even when the confounding signal is present in 100% of the training data. By training neural networks on these adversarial examples,we can improve their generalisation in collider bias settings. Experiments show state-of-the-art performance of LAD in label-free debiasing with gains of 76.12% on background coloured MNIST, 35.47% on fore-ground coloured MNIST, and 8.27% on corrupted CIFAR-10.
arXiv comment10 pages, 4 figures, submitted to AISTATS 2021
References in corpus (10)
- Neural Discrete Representation Learning
- Shortcut Learning in Deep Neural Networks
- Invariant Risk Minimization
- Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
- Exploring the Landscape of Spatial Robustness
- Measuring the tendency of CNNs to Learn Surface Statistical Regularities
- Adversarial Filters of Dataset Biases
- Learning De-biased Representations with Biased Representations
- In Search of Lost Domain Generalization
- Learning from Failure: Training Debiased Classifier from Biased Classifier