Deep Stable Learning for Out-Of-Distribution Generalization
arXiv:2104.07876
Abstract
Approaches based on deep neural networks have achieved striking performance when testing data and training data share similar distribution, but can significantly fail otherwise. Therefore, eliminating the impact of distribution shifts between training and testing data is crucial for building performance-promising deep models. Conventional methods assume either the known heterogeneity of training data (e.g. domain labels) or the approximately equal capacities of different domains. In this paper, we consider a more challenging case where neither of the above assumptions holds. We propose to address this problem by removing the dependencies between features via learning weights for training samples, which helps deep models get rid of spurious correlations and, in turn, concentrate more on the true connection between discriminative features and labels. Extensive experiments clearly demonstrate the effectiveness of our method on multiple distribution generalization benchmarks compared with state-of-the-art counterparts. Through extensive experiments on distribution generalization benchmarks including PACS, VLCS, MNIST-M, and NICO, we show the effectiveness of our method compared with state-of-the-art counterparts.
References in corpus (15)
- SmoothGrad: removing noise by adding noise
- Domain Generalization via Invariant Feature Representation
- Sanity Checks for Saliency Maps
- Domain Generalization via Model-Agnostic Learning of Semantic Features
- Deep Learning: A Critical Appraisal
- Generalizing Across Domains via Cross-Gradient Training
- Invariant Risk Minimization
- Out-of-Distribution Generalization via Risk Extrapolation (REx)
- Exploring the Landscape of Spatial Robustness
- A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
- Learning to Generalize: Meta-Learning for Domain Generalization
- Feature Alignment and Restoration for Domain Generalization and Adaptation
- Learning to Optimize Domain Specific Normalization for Domain Generalization
- Independently Interpretable Lasso: A New Regularizer for Sparse Regression with Uncorrelated Variables
- Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization