The Role of Embedding Complexity in Domain-invariant Representations

Abstract

Unsupervised domain adaptation aims to generalize the hypothesis trained in a source domain to an unlabeled target domain. One popular approach to this problem is to learn domain-invariant embeddings for both domains. In this work, we study, theoretically and empirically, the effect of the embedding complexity on generalization to the target domain. In particular, this complexity affects an upper bound on the target risk; this is reflected in experiments, too. Next, we specify our theoretical framework to multilayer neural networks. As a result, we develop a strategy that mitigates sensitivity to the embedding complexity, and empirically achieves performance on par with or better than the best layer-dependent complexity tradeoff.

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Unsupervised Domain Adaptation by Backpropagation
• Conditional Adversarial Domain Adaptation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Domain Separation Networks
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• On Learning Invariant Representation for Domain Adaptation
• Minimax Statistical Learning with Wasserstein Distances
• Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment
• Support and Invertibility in Domain-Invariant Representations