Domain Generalization: A Survey

Abstract

Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce. This is because most learning algorithms strongly rely on the i.i.d.~assumption on source/target data, which is often violated in practice due to domain shift. Domain generalization (DG) aims to achieve OOD generalization by using only source data for model learning. Over the last ten years, research in DG has made great progress, leading to a broad spectrum of methodologies, e.g., those based on domain alignment, meta-learning, data augmentation, or ensemble learning, to name a few; DG has also been studied in various application areas including computer vision, speech recognition, natural language processing, medical imaging, and reinforcement learning. In this paper, for the first time a comprehensive literature review in DG is provided to summarize the developments over the past decade. Specifically, we first cover the background by formally defining DG and relating it to other relevant fields like domain adaptation and transfer learning. Then, we conduct a thorough review into existing methods and theories. Finally, we conclude this survey with insights and discussions on future research directions.

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022

References in corpus (17)

• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Learning to Prompt for Vision-Language Models
• Improved Regularization of Convolutional Neural Networks with Cutout
• VisDA: The Visual Domain Adaptation Challenge
• Learn Convolutional Neural Network for Face Anti-Spoofing
• Generalized Out-of-Distribution Detection: A Survey
• MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• MEMO: Test Time Robustness via Adaptation and Augmentation
• Frustratingly Simple Domain Generalization via Image Stylization
• Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
• Feature Alignment and Restoration for Domain Generalization and Adaptation
• Improving Out-of-Distribution Robustness via Selective Augmentation
• A Generalization Error Bound for Multi-class Domain Generalization
• Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations
• On the Limitations of General Purpose Domain Generalisation Methods
• Latent Domain Learning with Dynamic Residual Adapters

Cited by in corpus (73)

• Learning to Prompt for Vision-Language Models
• A Survey of Human-in-the-loop for Machine Learning
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
• A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
• Domain Generalization for Medical Image Analysis: A Review
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture
• On The Effects Of Data Normalisation For Domain Adaptation On EEG Data
• From SLAM to Situational Awareness: Challenges and Survey
• SAN-Net: Learning Generalization to Unseen Sites for Stroke Lesion Segmentation with Self-Adaptive Normalization
• Dynamic Instance Domain Adaptation
• Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources
• Efficient Neural Neighborhood Search for Pickup and Delivery Problems
• Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement
• A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentation
• Hierarchical Disentanglement-Alignment Network for Robust SAR Vehicle Recognition
• Federated Domain Generalization: A Survey
• Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training
• Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification
• Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector
• Causal invariant geographic network representations with feature and structural distribution shifts
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• The Federated Tumor Segmentation (FeTS) Challenge
• Archangel: A Hybrid UAV-based Human Detection Benchmark with Position and Pose Metadata
• DGMamba: Domain Generalization via Generalized State Space Model
• FetMRQC: a robust quality control system for multi-centric fetal brain MRI
• Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization
• PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization
• Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)
• Data Heterogeneity Modeling for Trustworthy Machine Learning
• Learning to Generalize towards Unseen Domains via a Content-Aware Style Invariant Model for Disease Detection from Chest X-rays
• NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation
• Detecting Domain Shift in Multiple Instance Learning for Digital Pathology Using Fréchet Domain Distance
• Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization
• Is Machine Learning Unsafe and Irresponsible in Social Sciences? Paradoxes and Reconsidering from Recidivism Prediction Tasks
• Towards Hardware Supported Domain Generalization in DNN-Based Edge Computing Devices for Health Monitoring
• Transfer Learning Study of Motion Transformer-based Trajectory Predictions
• Representing Noisy Image Without Denoising
• Simple Domain Generalization Methods are Strong Baselines for Open Domain Generalization
• Phrase Grounding-based Style Transfer for Single-Domain Generalized Object Detection
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks
• Experts' cognition-driven ensemble deep learning for external validation of predicting pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer
• On the Generalization Properties of Deep Learning for Aircraft Fuel Flow Estimation Models
• HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization
• Pretrained Embeddings for E-commerce Machine Learning: When it Fails and Why?
• CNN Feature Map Augmentation for Single-Source Domain Generalization
• Hybrid Quantum-inspired Resnet and Densenet for Pattern Recognition
• Domain generalization of 3D semantic segmentation in autonomous driving
• Domain-randomized deep learning for neuroimage analysis
• Barycentric-alignment and reconstruction loss minimization for domain generalization
• DREAM: Domain-agnostic Reverse Engineering Attributes of Black-box Model
• Neural Posterior Estimation for Cataloging Astronomical Images with Spatially Varying Backgrounds and Point Spread Functions
• Domain Generalization for Endoscopic Image Segmentation by Disentangling Style-Content Information and SuperPixel Consistency
• Cross-domain Transfer of defect features in technical domains based on partial target data
• Augmentation-based Domain Generalization and Joint Training from Multiple Source Domains for Whole Heart Segmentation
• Unsupervised Domain Adaptation for Constraining Star Formation Histories
• Structure-preserving contrastive learning for spatial time series
• LA-CaRe-CNN: Cascading Refinement CNN for Left Atrial Scar Segmentation
• Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living
• GeMID: Generalizable Models for IoT Device Identification
• Space-scale Exploration of the Poor Reliability of Deep Learning Models: the Case of the Remote Sensing of Rooftop Photovoltaic Systems
• Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning
• Should All Noises Be Treated Equally: Impact of Input Noise Variability on Neural Network Robustness
• Generalized super-resolution 4D Flow MRI $\unicode{x2013}$ using ensemble learning to extend across the cardiovascular system
• Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
• SPACE: Semantic Projection and Alignment of CLIP Embeddings for Domain Adaptation
• Bringing Balance to Hand Shape Classification: Mitigating Data Imbalance Through Generative Models