Learning Task Grouping and Overlap in Multi-task Learning

Abstract

In the paradigm of multi-task learning, mul- tiple related prediction tasks are learned jointly, sharing information across the tasks. We propose a framework for multi-task learn- ing that enables one to selectively share the information across the tasks. We assume that each task parameter vector is a linear combi- nation of a finite number of underlying basis tasks. The coefficients of the linear combina- tion are sparse in nature and the overlap in the sparsity patterns of two tasks controls the amount of sharing across these. Our model is based on on the assumption that task pa- rameters within a group lie in a low dimen- sional subspace but allows the tasks in differ- ent groups to overlap with each other in one or more bases. Experimental results on four datasets show that our approach outperforms competing methods.

Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012)

References in corpus (4)

• Theoretical Models of Learning to Learn
• Multi-Task Feature Learning Via Efficient l2,1-Norm Minimization
• Bayesian Multitask Learning with Latent Hierarchies
• Flexible Modeling of Latent Task Structures in Multitask Learning

Cited by in corpus (31)

• Multi-Task Learning for Dense Prediction Tasks: A Survey
• Intelligent Traffic Monitoring Systems for Vehicle Classification: A Survey
• Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton
• A PAC-Bayesian bound for Lifelong Learning
• Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation
• MEDIC: A Multi-Task Learning Dataset for Disaster Image Classification
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Online Coreset Selection for Rehearsal-based Continual Learning
• Flexible Modeling of Latent Task Structures in Multitask Learning
• On Multiplicative Multitask Feature Learning
• Federated Continual Learning with Weighted Inter-client Transfer
• Variable Selection and Task Grouping for Multi-Task Learning
• Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret
• Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
• Continual Learning in Neural Networks
• Lifelong Learning of Compositional Structures
• Multi-task neural networks by learned contextual inputs
• Few-shot Sequence Learning with Transformers
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction
• Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
• Deep Multi-Task Learning via Generalized Tensor Trace Norm
• Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains
• The Traveling Observer Model: Multi-task Learning Through Spatial Variable Embeddings
• Tasks Structure Regularization in Multi-Task Learning for Improving Facial Attribute Prediction
• Unsupervised Domain Attention Adaptation Network for Caricature Attribute Recognition
• Learning Personalized Attribute Preference via Multi-task AUC Optimization
• Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning
• Compressed Hierarchical Representations for Multi-Task Learning and Task Clustering
• Deep Multi-Task Augmented Feature Learning via Hierarchical Graph Neural Network
• Latent Group Structured Multi-task Learning
• Towards Explainable Fact Checking