Clustered Multi-Task Learning: A Convex Formulation

Abstract

In multi-task learning several related tasks are considered simultaneously, with the hope that by an appropriate sharing of information across tasks, each task may benefit from the others. In the context of learning linear functions for supervised classification or regression, this can be achieved by including a priori information about the weight vectors associated with the tasks, and how they are expected to be related to each other. In this paper, we assume that tasks are clustered into groups, which are unknown beforehand, and that tasks within a group have similar weight vectors. We design a new spectral norm that encodes this a priori assumption, without the prior knowledge of the partition of tasks into groups, resulting in a new convex optimization formulation for multi-task learning. We show in simulations on synthetic examples and on the IEDB MHC-I binding dataset, that our approach outperforms well-known convex methods for multi-task learning, as well as related non convex methods dedicated to the same problem.

References in corpus (1)

• Low-rank matrix factorization with attributes

Cited by in corpus (48)

• Multi-Task Learning for Dense Prediction Tasks: A Survey
• A Survey on Multi-Task Learning
• GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
• Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
• Multi-Target Regression via Input Space Expansion: Treating Targets as Inputs
• AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning
• A Convex Formulation for Learning Task Relationships in Multi-Task Learning
• Federated Multi-Task Learning
• Distributed Clustering and Learning Over Networks
• Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning
• On-edge Multi-task Transfer Learning: Model and Practice with Data-driven Task Allocation
• Multitask diffusion adaptation over networks with common latent representations
• Multi-task Regression using Minimal Penalties
• Subspace Regularizers for Few-Shot Class Incremental Learning
• Convex Multitask Learning with Flexible Task Clusters
• On Multiplicative Multitask Feature Learning
• Variable Selection and Task Grouping for Multi-Task Learning
• Decentralized Clustering and Linking by Networked Agents
• Consistent Multitask Learning with Nonlinear Output Relations
• Implicit Model Specialization through DAG-based Decentralized Federated Learning
• Prediction of hierarchical time series using structured regularization and its application to artificial neural networks
• Multi-Task Averaging
• Efficient Output Kernel Learning for Multiple Tasks
• Learning to Multitask
• Scaled subordinators and generalizations of the Indian buffet process
• Unified View of Matrix Completion under General Structural Constraints
• Efficient Multitask Feature and Relationship Learning
• Byzantine Resilient Distributed Multi-Task Learning
• Topological Learning for Motion Data via Mixed Coordinates
• Finding the largest low-rank clusters with Ky Fan $2$-$k$-norm and ${\ell }_1$-norm
• Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
• DeepTag: inferring all-cause diagnoses from clinical notes in under-resourced medical domain
• Learning Leading Indicators for Time Series Predictions
• Framework for Multi-task Multiple Kernel Learning and Applications in Genome Analysis
• A Riemannian gossip approach to subspace learning on Grassmann manifold
• Co-evolutionary multi-task learning for dynamic time series prediction
• Distributed Multi-Task Relationship Learning
• Multi-task Supervised Learning via Cross-learning
• Multitask Learning for Sequence Labeling Tasks
• A Unifying Framework for Typical Multi-Task Multiple Kernel Learning Problems
• Towards Explainable Fact Checking
• Probabilistic task modelling for meta-learning
• Learning task structure via sparsity grouped multitask learning
• Transformative Machine Learning
• Simultaneous Parameter Learning and Bi-Clustering for Multi-Response Models
• Multi-Task Learning with Group-Specific Feature Space Sharing
• Electricity Demand Forecasting by Multi-Task Learning
• Learning with Clustering Structure