Hypothesis Transfer Learning via Transformation Functions

Abstract

We consider the Hypothesis Transfer Learning (HTL) problem where one incorporates a hypothesis trained on the source domain into the learning procedure of the target domain. Existing theoretical analysis either only studies specific algorithms or only presents upper bounds on the generalization error but not on the excess risk. In this paper, we propose a unified algorithm-dependent framework for HTL through a novel notion of transformation function, which characterizes the relation between the source and the target domains. We conduct a general risk analysis of this framework and in particular, we show for the first time, if two domains are related, HTL enjoys faster convergence rates of excess risks for Kernel Smoothing and Kernel Ridge Regression than those of the classical non-transfer learning settings. Experiments on real world data demonstrate the effectiveness of our framework.

Accepted by NIPS 2017

Cited by in corpus (13)

• Scalable Transfer Learning with Expert Models
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• On Inductive Biases for Heterogeneous Treatment Effect Estimation
• Theoretical Guarantees of Transfer Learning
• A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?
• On the Power of Multitask Representation Learning in Linear MDP
• Model Reuse with Reduced Kernel Mean Embedding Specification
• On Localized Discrepancy for Domain Adaptation
• Representation Learning Beyond Linear Prediction Functions
• Hypothesis Disparity Regularized Mutual Information Maximization
• AlphaNet: Improving Long-Tail Classification By Combining Classifiers
• Dynamic Knowledge Distillation for Black-box Hypothesis Transfer Learning
• Camera On-boarding for Person Re-identification using Hypothesis Transfer Learning