Improving out-of-distribution generalization via multi-task self-supervised pretraining

Abstract

Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained using self-supervised learning are comparable to, or better than, supervised learning for domain generalization in computer vision. We introduce a new self-supervised pretext task of predicting responses to Gabor filter banks and demonstrate that multi-task learning of compatible pretext tasks improves domain generalization performance as compared to training individual tasks alone. Features learnt through self-supervision obtain better generalization to unseen domains when compared to their supervised counterpart when there is a larger domain shift between training and test distributions and even show better localization ability for objects of interest. Self-supervised feature representations can also be combined with other domain generalization methods to further boost performance.

References in corpus (6)

• Representation Learning with Contrastive Predictive Coding
• Momentum Contrast for Unsupervised Visual Representation Learning
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Invariant Risk Minimization
• A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
• When Does Self-supervision Improve Few-shot Learning?

Cited by in corpus (11)

• Domain Generalization: A Survey
• Generalizing to unseen domains via distribution matching
• In Search of Lost Domain Generalization
• Domain Generalization using Causal Matching
• 3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection
• Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
• Improving colonoscopy lesion classification using semi-supervised deep learning
• MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaption
• POEM: Polarization of Embeddings for Domain-Invariant Representations
• Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation
• Pareto Navigation Gradient Descent: a First-Order Algorithm for Optimization in Pareto Set