Learning to Learn with Variational Information Bottleneck for Domain Generalization

Abstract

Domain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift. In this paper, we address both problems. We introduce a probabilistic meta-learning model for domain generalization, in which classifier parameters shared across domains are modeled as distributions. This enables better handling of prediction uncertainty on unseen domains. To deal with domain shift, we learn domain-invariant representations by the proposed principle of meta variational information bottleneck, we call MetaVIB. MetaVIB is derived from novel variational bounds of mutual information, by leveraging the meta-learning setting of domain generalization. Through episodic training, MetaVIB learns to gradually narrow domain gaps to establish domain-invariant representations, while simultaneously maximizing prediction accuracy. We conduct experiments on three benchmarks for cross-domain visual recognition. Comprehensive ablation studies validate the benefits of MetaVIB for domain generalization. The comparison results demonstrate our method outperforms previous approaches consistently.

15 pages, 4 figures, ECCV2020

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• The information bottleneck method
• Theoretical Models of Learning to Learn
• Opening the Black Box of Deep Neural Networks via Information
• Domain Separation Networks
• Domain Generalization via Invariant Feature Representation
• On First-Order Meta-Learning Algorithms
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Few-Shot Learning with Graph Neural Networks
• Generalizing Across Domains via Cross-Gradient Training
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
• Feature-Critic Networks for Heterogeneous Domain Generalization
• Learning to Learn Kernels with Variational Random Features

Cited by in corpus (1)

• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space