Learning to Optimize Domain Specific Normalization for Domain Generalization

Abstract

We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers that are specific to individual domains. Our approach employs multiple normalization methods while learning separate affine parameters per domain. For each domain, the activations are normalized by a weighted average of multiple normalization statistics. The normalization statistics are kept track of separately for each normalization type if necessary. Specifically, we employ batch and instance normalizations in our implementation to identify the best combination of these two normalization methods in each domain. The optimized normalization layers are effective to enhance the generalizability of the learned model. We demonstrate the state-of-the-art accuracy of our algorithm in the standard domain generalization benchmarks, as well as viability to further tasks such as multi-source domain adaptation and domain generalization in the presence of label noise.

References in corpus (9)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Spectral Normalization for Generative Adversarial Networks
• Domain Generalization via Invariant Feature Representation
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Differentiable Learning-to-Normalize via Switchable Normalization
• Universal representations:The missing link between faces, text, planktons, and cat breeds
• Moment Matching for Multi-Source Domain Adaptation
• Domain Generalization via Conditional Invariant Representation

Cited by in corpus (8)

• Deep Stable Learning for Out-Of-Distribution Generalization
• Domain Invariant Representation Learning with Domain Density Transformations
• Meta Batch-Instance Normalization for Generalizable Person Re-Identification
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
• Domain Generalization via Inference-time Label-Preserving Target Projections
• Decentralised Person Re-Identification with Selective Knowledge Aggregation
• Generalizable Model-agnostic Semantic Segmentation via Target-specific Normalization
• Discriminative Domain-Invariant Adversarial Network for Deep Domain Generalization