Domain Generalization: A Survey

Abstract

Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce. This is because most learning algorithms strongly rely on the i.i.d.~assumption on source/target data, which is often violated in practice due to domain shift. Domain generalization (DG) aims to achieve OOD generalization by using only source data for model learning. Over the last ten years, research in DG has made great progress, leading to a broad spectrum of methodologies, e.g., those based on domain alignment, meta-learning, data augmentation, or ensemble learning, to name a few; DG has also been studied in various application areas including computer vision, speech recognition, natural language processing, medical imaging, and reinforcement learning. In this paper, for the first time a comprehensive literature review in DG is provided to summarize the developments over the past decade. Specifically, we first cover the background by formally defining DG and relating it to other relevant fields like domain adaptation and transfer learning. Then, we conduct a thorough review into existing methods and theories. Finally, we conclude this survey with insights and discussions on future research directions.

arXiv commentIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022

References in corpus (33)

• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Learning to Prompt for Vision-Language Models
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Improved Regularization of Convolutional Neural Networks with Cutout
• Shortcut Learning in Deep Neural Networks
• Recent Advances in Open Set Recognition: A Survey
• VisDA: The Visual Domain Adaptation Challenge
• Learn Convolutional Neural Network for Face Anti-Spoofing
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Generalized Out-of-Distribution Detection: A Survey
• MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data
• Tent: Fully Test-time Adaptation by Entropy Minimization
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Domain Adaptive Ensemble Learning
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Domain Generalization by Marginal Transfer Learning
• Improving the generalizability of convolutional neural network-based segmentation on CMR images
• MEMO: Test Time Robustness via Adaptation and Augmentation
• Generalizing to unseen domains via distribution matching
• Domain Generalization using Causal Matching
• Frustratingly Simple Domain Generalization via Image Stylization
• Robust Place Categorization with Deep Domain Generalization
• Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
• Feature Alignment and Restoration for Domain Generalization and Adaptation
• Improving out-of-distribution generalization via multi-task self-supervised pretraining
• Improving Out-of-Distribution Robustness via Selective Augmentation
• A Generalization Error Bound for Multi-class Domain Generalization
• Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations
• On the Limitations of General Purpose Domain Generalisation Methods
• Latent Domain Learning with Dynamic Residual Adapters

Cited by in corpus (74)

• Learning to Prompt for Vision-Language Models
• A Survey of Human-in-the-loop for Machine Learning
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
• A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
• Domain Generalization for Medical Image Analysis: A Review
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• On The Effects Of Data Normalisation For Domain Adaptation On EEG Data
• Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture
• From SLAM to Situational Awareness: Challenges and Survey
• SAN-Net: Learning Generalization to Unseen Sites for Stroke Lesion Segmentation with Self-Adaptive Normalization
• Dynamic Instance Domain Adaptation
• Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources
• Efficient Neural Neighborhood Search for Pickup and Delivery Problems
• Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement
• Hierarchical Disentanglement-Alignment Network for Robust SAR Vehicle Recognition
• A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentation
• Federated Domain Generalization: A Survey
• Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks
• M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification
• Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector
• Causal invariant geographic network representations with feature and structural distribution shifts
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• Archangel: A Hybrid UAV-based Human Detection Benchmark with Position and Pose Metadata
• The Federated Tumor Segmentation (FeTS) Challenge
• FetMRQC: a robust quality control system for multi-centric fetal brain MRI
• DGMamba: Domain Generalization via Generalized State Space Model
• Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization
• PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization
• Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)
• Data Heterogeneity Modeling for Trustworthy Machine Learning
• Learning to Generalize towards Unseen Domains via a Content-Aware Style Invariant Model for Disease Detection from Chest X-rays
• NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation
• Is Machine Learning Unsafe and Irresponsible in Social Sciences? Paradoxes and Reconsidering from Recidivism Prediction Tasks
• Detecting Domain Shift in Multiple Instance Learning for Digital Pathology Using Fréchet Domain Distance
• Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization
• Representing Noisy Image Without Denoising
• Transfer Learning Study of Motion Transformer-based Trajectory Predictions
• Towards Hardware Supported Domain Generalization in DNN-Based Edge Computing Devices for Health Monitoring
• Simple Domain Generalization Methods are Strong Baselines for Open Domain Generalization
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• Phrase Grounding-based Style Transfer for Single-Domain Generalized Object Detection
• A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks
• Experts' cognition-driven ensemble deep learning for external validation of predicting pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer
• CNN Feature Map Augmentation for Single-Source Domain Generalization
• HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization
• Pretrained Embeddings for E-commerce Machine Learning: When it Fails and Why?
• Hybrid Quantum-inspired Resnet and Densenet for Pattern Recognition
• Domain generalization of 3D semantic segmentation in autonomous driving
• On the Generalization Properties of Deep Learning for Aircraft Fuel Flow Estimation Models
• Domain-randomized deep learning for neuroimage analysis
• Barycentric-alignment and reconstruction loss minimization for domain generalization
• Augmentation-based Domain Generalization and Joint Training from Multiple Source Domains for Whole Heart Segmentation
• Neural Posterior Estimation for Cataloging Astronomical Images with Spatially Varying Backgrounds and Point Spread Functions
• Cross-domain Transfer of defect features in technical domains based on partial target data
• Domain Generalization for Endoscopic Image Segmentation by Disentangling Style-Content Information and SuperPixel Consistency
• DREAM: Domain-agnostic Reverse Engineering Attributes of Black-box Model
• Unsupervised Domain Adaptation for Constraining Star Formation Histories
• Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living
• LA-CaRe-CNN: Cascading Refinement CNN for Left Atrial Scar Segmentation
• Space-scale Exploration of the Poor Reliability of Deep Learning Models: the Case of the Remote Sensing of Rooftop Photovoltaic Systems
• Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning
• GeMID: Generalizable Models for IoT Device Identification
• Structure-preserving contrastive learning for spatial time series
• Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
• Should All Noises Be Treated Equally: Impact of Input Noise Variability on Neural Network Robustness
• Bringing Balance to Hand Shape Classification: Mitigating Data Imbalance Through Generative Models
• Does anatomical contextual information improve 3D U-Net based brain tumor segmentation?
• SPACE: Semantic Projection and Alignment of CLIP Embeddings for Domain Adaptation
• Generalized super-resolution 4D Flow MRI $\unicode{x2013}$ using ensemble learning to extend across the cardiovascular system