Learning Debiased and Disentangled Representations for Semantic Segmentation
arXiv:2111.00531
Abstract
Deep neural networks are susceptible to learn biased models with entangled feature representations, which may lead to subpar performances on various downstream tasks. This is particularly true for under-represented classes, where a lack of diversity in the data exacerbates the tendency. This limitation has been addressed mostly in classification tasks, but there is little study on additional challenges that may appear in more complex dense prediction problems including semantic segmentation. To this end, we propose a model-agnostic and stochastic training scheme for semantic segmentation, which facilitates the learning of debiased and disentangled representations. For each class, we first extract class-specific information from the highly entangled feature map. Then, information related to a randomly sampled class is suppressed by a feature selection process in the feature space. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes, and the model is able to learn more debiased and disentangled feature representations. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks, with especially notable performance gains on under-represented classes.
Accepted by NeurIPS 2021
References in corpus (12)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- Shortcut Learning in Deep Neural Networks
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
- Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping
- Learning De-biased Representations with Biased Representations
- Learning Robust Representations by Projecting Superficial Statistics Out
- No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems
- Analyzing and Improving Representations with the Soft Nearest Neighbor Loss
- Unsupervised Learning of Debiased Representations with Pseudo-Attributes