Distributed Stochastic Multi-Task Learning with Graph Regularization

Abstract

We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines. Uniform averaging or diminishing stepsize in these methods would yield consensus (single task) learning. We show how simply skewing the averaging weights or controlling the stepsize allows learning different, but related, tasks on the different machines.

References in corpus (4)

• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Optimal algorithms for smooth and strongly convex distributed optimization in networks
• Distributed Multi-Task Learning with Shared Representation
• Distributed Multitask Learning

Cited by in corpus (8)

• Federated Learning of a Mixture of Global and Local Models
• Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning
• Lower Bounds and Optimal Algorithms for Personalized Federated Learning
• Minimax Estimation for Personalized Federated Learning: An Alternative between FedAvg and Local Training?
• Decentralized Multi-Task Learning Based on Extreme Learning Machines
• Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
• Distributed Linear Model Clustering over Networks: A Tree-Based Fused-Lasso ADMM Approach
• GT-STORM: Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning