A New PAC-Bayesian Perspective on Domain Adaptation

Abstract

We study the issue of PAC-Bayesian domain adaptation: We want to learn, from a source domain, a majority vote model dedicated to a target one. Our theoretical contribution brings a new perspective by deriving an upper-bound on the target risk where the distributions' divergence---expressed as a ratio---controls the trade-off between a source error measure and the target voters' disagreement. Our bound suggests that one has to focus on regions where the source data is informative.From this result, we derive a PAC-Bayesian generalization bound, and specialize it to linear classifiers. Then, we infer a learning algorithmand perform experiments on real data.

Published at ICML 2016

References in corpus (5)

• Frustratingly Easy Domain Adaptation
• Marginalized Denoising Autoencoders for Domain Adaptation
• A Note on the PAC Bayesian Theorem
• Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm
• A New PAC-Bayesian Perspective on Domain Adaptation

Cited by in corpus (14)

• A review of domain adaptation without target labels
• An introduction to domain adaptation and transfer learning
• User-friendly introduction to PAC-Bayes bounds
• A Primer on PAC-Bayesian Learning
• A New PAC-Bayesian Perspective on Domain Adaptation
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• PAC-Bayes and Domain Adaptation
• Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
• A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptation
• On Localized Discrepancy for Domain Adaptation
• Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
• On Label Shift in Domain Adaptation via Wasserstein Distance
• Unifying Variational Inference and PAC-Bayes for Supervised Learning that Scales
• Self-Taught Support Vector Machine