PCA-based Multi Task Learning: a Random Matrix Approach
arXiv:2111.00924
Abstract
The article proposes and theoretically analyses a \emph{computationally efficient} multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes \cite{barshan2011supervised,bair2006prediction}. The analysis reveals that (i) by default learning may dramatically fail by suffering from \emph{negative transfer}, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks show that the proposed method achieves comparable performance with state-of-the-art MTL methods but at a \emph{significantly reduced computational cost}.
References in corpus (7)
- An Overview of Multi-Task Learning in Deep Neural Networks
- Unsupervised Domain Adaptation with Residual Transfer Networks
- Multi-Task Feature Learning Via Efficient l2,1-Norm Minimization
- Quantifying the Carbon Emissions of Machine Learning
- Convergence and prediction of principal component scores in high-dimensional settings
- Unsupervised and Supervised Principal Component Analysis: Tutorial
- Two-way kernel matrix puncturing: towards resource-efficient PCA and spectral clustering