A survey on domain adaptation theory: learning bounds and theoretical guarantees

Abstract

All famous machine learning algorithms that comprise both supervised and semi-supervised learning work well only under a common assumption: the training and test data follow the same distribution. When the distribution changes, most statistical models must be reconstructed from newly collected data, which for some applications can be costly or impossible to obtain. Therefore, it has become necessary to develop approaches that reduce the need and the effort to obtain new labeled samples by exploiting data that are available in related areas, and using these further across similar fields. This has given rise to a new machine learning framework known as transfer learning: a learning setting inspired by the capability of a human being to extrapolate knowledge across tasks to learn more efficiently. Despite a large amount of different transfer learning scenarios, the main objective of this survey is to provide an overview of the state-of-the-art theoretical results in a specific, and arguably the most popular, sub-field of transfer learning, called domain adaptation. In this sub-field, the data distribution is assumed to change across the training and the test data, while the learning task remains the same. We provide a first up-to-date description of existing results related to domain adaptation problem that cover learning bounds based on different statistical learning frameworks.

References in corpus (15)

• Equivalence of distance-based and RKHS-based statistics in hypothesis testing
• Domain Adaptation: Learning Bounds and Algorithms
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• Bridging Theory and Algorithm for Domain Adaptation
• On Learning Invariant Representation for Domain Adaptation
• Algorithms and Theory for Multiple-Source Adaptation
• Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm
• Multiple Source Adaptation and the Renyi Divergence
• Fast Rates by Transferring from Auxiliary Hypotheses
• Hypothesis Transfer Learning via Transformation Functions
• A New PAC-Bayesian Perspective on Domain Adaptation
• PAC-Bayes and Domain Adaptation
• On the Value of Target Data in Transfer Learning
• Minimum Integrated Distance Estimation in Simultaneous Equation Models
• Optimal Transport for Domain Adaptation

Cited by in corpus (9)

• Lifelong Teacher-Student Network Learning
• Does Invariant Risk Minimization Capture Invariance?
• Domain adaptation under structural causal models
• Multi-Source Domain Adaptation through Dataset Dictionary Learning in Wasserstein Space
• Distributional Anchor Regression
• A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms
• Affine Transport for Sim-to-Real Domain Adaptation
• A Symmetric Loss Perspective of Reliable Machine Learning
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?