A Theory of Label Propagation for Subpopulation Shift
arXiv:2102.11203
Abstract
One of the central problems in machine learning is domain adaptation. Unlike past theoretical work, we consider a new model for subpopulation shift in the input or representation space. In this work, we propose a provably effective framework for domain adaptation based on label propagation. In our analysis, we use a simple but realistic expansion assumption, proposed in \citet{wei2021theoretical}. Using a teacher classifier trained on the source domain, our algorithm not only propagates to the target domain but also improves upon the teacher. By leveraging existing generalization bounds, we also obtain end-to-end finite-sample guarantees on the entire algorithm. In addition, we extend our theoretical framework to a more general setting of source-to-target transfer based on a third unlabeled dataset, which can be easily applied in various learning scenarios. Inspired by our theory, we adapt consistency-based semi-supervised learning methods to domain adaptation settings and gain significant improvements.
arXiv commentICML 2021
References in corpus (22)
- Learning Transferable Features with Deep Adaptation Networks
- Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
- Conditional Adversarial Domain Adaptation
- A DIRT-T Approach to Unsupervised Domain Adaptation
- Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
- Invariant Risk Minimization
- Out-of-Distribution Generalization via Risk Extrapolation (REx)
- Domain-Adversarial Neural Networks
- Bridging Theory and Algorithm for Domain Adaptation
- On Learning Invariant Representation for Domain Adaptation
- Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
- In Search of Lost Domain Generalization
- Multi-source Domain Adaptation for Semantic Segmentation
- Learning explanations that are hard to vary
- Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment
- Generative Adversarial Nets from a Density Ratio Estimation Perspective
- Towards a Theoretical Framework of Out-of-Distribution Generalization
- Representation Learning via Invariant Causal Mechanisms
- Rethinking Distributional Matching Based Domain Adaptation
- Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
- Learning Causal Models Online
- Fundamental Limits and Tradeoffs in Invariant Representation Learning