Unsupervised Learning of Debiased Representations with Pseudo-Attributes
arXiv:2108.02943
Abstract
Dataset bias is a critical challenge in machine learning since it often leads to a negative impact on a model due to the unintended decision rules captured by spurious correlations. Although existing works often handle this issue based on human supervision, the availability of the proper annotations is impractical and even unrealistic. To better tackle the limitation, we propose a simple but effective unsupervised debiasing technique. Specifically, we first identify pseudo-attributes based on the results from clustering performed in the feature embedding space even without an explicit bias attribute supervision. Then, we employ a novel cluster-wise reweighting scheme to learn debiased representation; the proposed method prevents minority groups from being discounted for minimizing the overall loss, which is desirable for worst-case generalization. The extensive experiments demonstrate the outstanding performance of our approach on multiple standard benchmarks, even achieving the competitive accuracy to the supervised counterpart.
References in corpus (15)
- Shortcut Learning in Deep Neural Networks
- ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
- Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
- Fair Resource Allocation in Federated Learning
- Agnostic Federated Learning
- The Frontiers of Fairness in Machine Learning
- Women also Snowboard: Overcoming Bias in Captioning Models
- Does Distributionally Robust Supervised Learning Give Robust Classifiers?
- Learning De-biased Representations with Biased Representations
- Learning Robust Representations by Projecting Superficial Statistics Out
- An Investigation of Why Overparameterization Exacerbates Spurious Correlations
- Learning from Failure: Training Debiased Classifier from Biased Classifier
- No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems
- Distributionally Robust Bayesian Optimization
- Towards Accuracy-Fairness Paradox: Adversarial Example-based Data Augmentation for Visual Debiasing