PAC-Bayes and Domain Adaptation
arXiv:1707.05712 · doi:10.1016/j.neucom.2019.10.105
Abstract
We provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different, but related, target distribution. Firstly, we propose an improvement of the previous approach we proposed in Germain et al. (2013), which relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter domain adaptation bound for the target risk. While this bound stands in the spirit of common domain adaptation works, we derive a second bound (introduced in Germain et al., 2016) that brings a new perspective on domain adaptation by deriving an upper bound on the target risk where the distributions' divergence-expressed as a ratio-controls the trade-off between a source error measure and the target voters' disagreement. We discuss and compare both results, from which we obtain PAC-Bayesian generalization bounds. Furthermore, from the PAC-Bayesian specialization to linear classifiers, we infer two learning algorithms, and we evaluate them on real data.
Neurocomputing, Elsevier, 2019. arXiv admin note: substantial text overlap with arXiv:1503.06944
References in corpus (12)
- Deep Visual Domain Adaptation: A Survey
- Marginalized Denoising Autoencoders for Domain Adaptation
- A DIRT-T Approach to Unsupervised Domain Adaptation
- Algorithms and Theory for Multiple-Source Adaptation
- A PAC-Bayesian bound for Lifelong Learning
- A Note on the PAC Bayesian Theorem
- Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm
- Euclidean Distances, soft and spectral Clustering on Weighted Graphs
- Multiple Source Adaptation and the Renyi Divergence
- A New PAC-Bayesian Perspective on Domain Adaptation
- Domain Adaptation of Majority Votes via Perturbed Variation-based Label Transfer
- PAC-Bayesian Theorems for Domain Adaptation with Specialization to Linear Classifiers