A Style and Semantic Memory Mechanism for Domain Generalization

Abstract

Mainstream state-of-the-art domain generalization algorithms tend to prioritize the assumption on semantic invariance across domains. Meanwhile, the inherent intra-domain style invariance is usually underappreciated and put on the shelf. In this paper, we reveal that leveraging intra-domain style invariance is also of pivotal importance in improving the efficiency of domain generalization. We verify that it is critical for the network to be informative on what domain features are invariant and shared among instances, so that the network sharpens its understanding and improves its semantic discriminative ability. Correspondingly, we also propose a novel "jury" mechanism, which is particularly effective in learning useful semantic feature commonalities among domains. Our complete model called STEAM can be interpreted as a novel probabilistic graphical model, for which the implementation requires convenient constructions of two kinds of memory banks: semantic feature bank and style feature bank. Empirical results show that our proposed framework surpasses the state-of-the-art methods by clear margins.

arXiv commentICCV 2021

References in corpus (7)

• A Simple Framework for Contrastive Learning of Visual Representations
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Conditional Adversarial Domain Adaptation
• Contrastive Multiview Coding
• Frustratingly Simple Domain Generalization via Image Stylization
• Domain Generalization with MixStyle
• Representation Learning via Invariant Causal Mechanisms