A Convex Formulation for Learning Task Relationships in Multi-Task Learning

Abstract

Multi-task learning is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other related tasks. In this paper, we propose a regularization formulation for learning the relationships between tasks in multi-task learning. This formulation can be viewed as a novel generalization of the regularization framework for single-task learning. Besides modeling positive task correlation, our method, called multi-task relationship learning (MTRL), can also describe negative task correlation and identify outlier tasks based on the same underlying principle. Under this regularization framework, the objective function of MTRL is convex. For efficiency, we use an alternating method to learn the optimal model parameters for each task as well as the relationships between tasks. We study MTRL in the symmetric multi-task learning setting and then generalize it to the asymmetric setting as well. We also study the relationships between MTRL and some existing multi-task learning methods. Experiments conducted on a toy problem as well as several benchmark data sets demonstrate the effectiveness of MTRL.

Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)

References in corpus (4)

• Multi-Task Feature Learning Via Efficient l2,1-Norm Minimization
• Clustered Multi-Task Learning: A Convex Formulation
• Bayesian Multitask Learning with Latent Hierarchies
• Bayesian Multitask Learning with Latent Hierarchies

Cited by in corpus (23)

• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning
• A Multi-task Framework for Infrared Small Target Detection and Segmentation
• Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Multi-task Sparse Structure Learning
• When Deep Learning Meets Multi-Task Learning in SAR ATR: Simultaneous Target Recognition and Segmentation
• SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks
• Scene Graph Generation from Objects, Phrases and Region Captions
• Collaborative Deep Reinforcement Learning
• Decentralized Multi-Task Learning Based on Extreme Learning Machines
• Efficient Multitask Feature and Relationship Learning
• Aggregating From Multiple Target-Shifted Sources
• Conditional Adversarial Networks for Multi-Domain Text Classification
• Cartoon Face Recognition: A Benchmark Dataset
• MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving
• Outlier Robust Extreme Learning Machine for Multi-Target Regression
• A Multi-task Two-stream Spatiotemporal Convolutional Neural Network for Convective Storm Nowcasting
• Framework for Multi-task Multiple Kernel Learning and Applications in Genome Analysis
• Multi-task Learning in the Computerized Diagnosis of Breast Cancer on DCE-MRIs
• Exploring Spatio-Temporal and Cross-Type Correlations for Crime Prediction
• Conic Multi-Task Classification
• Multi-Task Learning with Group-Specific Feature Space Sharing