A Bit More Bayesian: Domain-Invariant Learning with Uncertainty

Abstract

Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.

arXiv commentaccepted to ICML 2021

References in corpus (22)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Learning Transferable Features with Deep Adaptation Networks
• On Calibration of Modern Neural Networks
• Weight Uncertainty in Neural Networks
• Domain Generalization via Invariant Feature Representation
• Variational Dropout and the Local Reparameterization Trick
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Generalizing Across Domains via Cross-Gradient Training
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
• Learning Robust Representations by Projecting Superficial Statistics Out
• DIVA: Domain Invariant Variational Autoencoders
• In Search of Lost Domain Generalization
• Efficient Domain Generalization via Common-Specific Low-Rank Decomposition
• Domain Generalization with MixStyle
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
• Learning to Optimize Domain Specific Normalization for Domain Generalization
• Learning to Generate Novel Domains for Domain Generalization
• Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
• The Deep Weight Prior