DGMamba: Domain Generalization via Generalized State Space Model

Abstract

Domain generalization~(DG) aims at solving distribution shift problems in various scenes. Existing approaches are based on Convolution Neural Networks (CNNs) or Vision Transformers (ViTs), which suffer from limited receptive fields or quadratic complexities issues. Mamba, as an emerging state space model (SSM), possesses superior linear complexity and global receptive fields. Despite this, it can hardly be applied to DG to address distribution shifts, due to the hidden state issues and inappropriate scan mechanisms. In this paper, we propose a novel framework for DG, named DGMamba, that excels in strong generalizability toward unseen domains and meanwhile has the advantages of global receptive fields, and efficient linear complexity. Our DGMamba compromises two core components: Hidden State Suppressing~(HSS) and Semantic-aware Patch refining~(SPR). In particular, HSS is introduced to mitigate the influence of hidden states associated with domain-specific features during output prediction. SPR strives to encourage the model to concentrate more on objects rather than context, consisting of two designs: Prior-Free Scanning~(PFS), and Domain Context Interchange~(DCI). Concretely, PFS aims to shuffle the non-semantic patches within images, creating more flexible and effective sequences from images, and DCI is designed to regularize Mamba with the combination of mismatched non-semantic and semantic information by fusing patches among domains. Extensive experiments on five commonly used DG benchmarks demonstrate that the proposed DGMamba achieves remarkably superior results to state-of-the-art models. The code will be made publicly available at https://github.com/longshaocong/DGMamba.

Accepted to ACM MM 2024

References in corpus (12)

• Domain Generalization: A Survey
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Instance-Aware Domain Generalization for Face Anti-Spoofing
• Diagonal State Spaces are as Effective as Structured State Spaces
• Adaptive Mixture of Experts Learning for Generalizable Face Anti-Spoofing
• Partial success in closing the gap between human and machine vision
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation
• Domain Adaptive Semantic Segmentation via Regional Contrastive Consistency Regularization
• Rethinking Domain Generalization: Discriminability and Generalizability
• Adaptive Risk Minimization: Learning to Adapt to Domain Shift
• Mix and Reason: Reasoning over Semantic Topology with Data Mixing for Domain Generalization
• Rethinking Importance Weighting for Deep Learning under Distribution Shift