Few-shot Domain Adaptation by Causal Mechanism Transfer

Abstract

We study few-shot supervised domain adaptation (DA) for regression problems, where only a few labeled target domain data and many labeled source domain data are available. Many of the current DA methods base their transfer assumptions on either parametrized distribution shift or apparent distribution similarities, e.g., identical conditionals or small distributional discrepancies. However, these assumptions may preclude the possibility of adaptation from intricately shifted and apparently very different distributions. To overcome this problem, we propose mechanism transfer, a meta-distributional scenario in which a data generating mechanism is invariant among domains. This transfer assumption can accommodate nonparametric shifts resulting in apparently different distributions while providing a solid statistical basis for DA. We take the structural equations in causal modeling as an example and propose a novel DA method, which is shown to be useful both theoretically and experimentally. Our method can be seen as the first attempt to fully leverage the structural causal models for DA.

arXiv comment33 pages, 3 figures. Camera-ready version for Thirty-seventh International Conference on Machine Learning (ICML 2020)

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Invariant Risk Minimization
• Bridging Theory and Algorithm for Domain Adaptation
• Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
• Invariant Models for Causal Transfer Learning
• Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
• Semi-Supervised Domain Adaptation with Non-Parametric Copulas
• Causal Generative Domain Adaptation Networks
• Learning Bound for Parameter Transfer Learning

Cited by in corpus (11)

• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
• ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA
• Zero-Shot Recommender Systems
• Discrepancy-Based Active Learning for Domain Adaptation
• Latent Causal Invariant Model
• Few-shot Unsupervised Domain Adaptation with Image-to-class Sparse Similarity Encoding
• Dynamic Feature Alignment for Semi-supervised Domain Adaptation
• Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer
• TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation
• On the Statistical Benefits of Curriculum Learning