Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning

Abstract

The problem of learning simultaneously several related tasks has received considerable attention in several domains, especially in machine learning with the so-called multitask learning problem or learning to learn problem [1], [2]. Multitask learning is an approach to inductive transfer learning (using what is learned for one problem to assist in another problem) and helps improve generalization performance relative to learning each task separately by using the domain information contained in the training signals of related tasks as an inductive bias. Several strategies have been derived within this community under the assumption that all data are available beforehand at a fusion center. However, recent years have witnessed an increasing ability to collect data in a distributed and streaming manner. This requires the design of new strategies for learning jointly multiple tasks from streaming data over distributed (or networked) systems. This article provides an overview of multitask strategies for learning and adaptation over networks. The working hypothesis for these strategies is that agents are allowed to cooperate with each other in order to learn distinct, though related tasks. The article shows how cooperation steers the network limiting point and how different cooperation rules allow to promote different task relatedness models. It also explains how and when cooperation over multitask networks outperforms non-cooperative strategies.

References in corpus (14)

• Theoretical Models of Learning to Learn
• Multitask Diffusion Adaptation over Networks
• Clustered Multi-Task Learning: A Convex Formulation
• Diffusion LMS over Multitask Networks
• Network Lasso: Clustering and Optimization in Large Graphs
• A Convex Formulation for Learning Task Relationships in Multi-Task Learning
• Distributed Optimization With Local Domains: Applications in MPC and Network Flows
• Distributed Clustering and Learning Over Networks
• Distributed Diffusion-Based LMS for Node-Specific Adaptive Parameter Estimation
• Proximity Without Consensus in Online Multi-Agent Optimization
• Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
• Graph-Structured Multi-task Regression and an Efficient Optimization Method for General Fused Lasso
• $\mathcal{C}\mathcal{I}\mathcal{R}\mathcal{FE}$: A Distributed Random Fields Estimator
• Distributed Stochastic Multi-Task Learning with Graph Regularization

Cited by in corpus (12)

• Graph signal processing for machine learning: A review and new perspectives
• Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems
• Quantization for decentralized learning under subspace constraints
• Diffusion LMS with Communication Delays: Stability and Performance Analysis
• Graph Laplacian Diffusion Localization of Connected and Automated Vehicles
• Networked Signal and Information Processing
• Transient Theoretical Analysis of Diffusion RLS Algorithm for Cyclostationary Colored Inputs
• Tracking Performance of Online Stochastic Learners
• Compressed Regression over Adaptive Networks
• Byzantine Resilient Distributed Multi-Task Learning
• Asynchronous Message-Passing and Zeroth-Order Optimization Based Distributed Learning with a Use-Case in Resource Allocation in Communication Networks
• On the Energy and Communication Efficiency Tradeoffs in Federated and Multi-Task Learning