Federated Multi-Task Learning

Abstract

Federated learning poses new statistical and systems challenges in training machine learning models over distributed networks of devices. In this work, we show that multi-task learning is naturally suited to handle the statistical challenges of this setting, and propose a novel systems-aware optimization method, MOCHA, that is robust to practical systems issues. Our method and theory for the first time consider issues of high communication cost, stragglers, and fault tolerance for distributed multi-task learning. The resulting method achieves significant speedups compared to alternatives in the federated setting, as we demonstrate through simulations on real-world federated datasets.

References in corpus (1)

• Clustered Multi-Task Learning: A Convex Formulation

Cited by in corpus (51)

• Federated Learning with Non-IID Data
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Deep Reinforcement Learning: An Overview
• Agnostic Federated Learning
• Personalized Federated Learning with Moreau Envelopes
• Federated Forest
• Edge Intelligence: Architectures, Challenges, and Applications
• Abnormal Client Behavior Detection in Federated Learning
• Privacy-preserving Artificial Intelligence Techniques in Biomedicine
• Flower: A Friendly Federated Learning Research Framework
• Communication Efficiency in Federated Learning: Achievements and Challenges
• FedBABU: Towards Enhanced Representation for Federated Image Classification
• FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
• Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions
• Fed-Focal Loss for imbalanced data classification in Federated Learning
• Personalized Federated Learning using Hypernetworks
• GIANT: Globally Improved Approximate Newton Method for Distributed Optimization
• FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
• FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• Communication-Efficient Edge AI: Algorithms and Systems
• FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion
• Fast Federated Learning in the Presence of Arbitrary Device Unavailability
• Federated Reconstruction: Partially Local Federated Learning
• Personalized Federated Learning through Local Memorization
• Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges
• FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update
• Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
• RingFed: Reducing Communication Costs in Federated Learning on Non-IID Data
• FedFm: Towards a Robust Federated Learning Approach For Fault Mitigation at the Edge Nodes
• Benchmarking Differential Privacy and Federated Learning for BERT Models
• Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
• A decentralized aggregation mechanism for training deep learning models using smart contract system for bank loan prediction
• FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoring
• Curie: Policy-based Secure Data Exchange
• Federated Multi-task Hierarchical Attention Model for Sensor Analytics
• Real-Time Decentralized knowledge Transfer at the Edge
• Federated Extra-Trees with Privacy Preserving
• New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning
• Demystifying the Effects of Non-Independence in Federated Learning
• Introducing Noise in Decentralized Training of Neural Networks
• Segmented Federated Learning for Adaptive Intrusion Detection System
• Distributed Primal-Dual Optimization for Online Multi-Task Learning
• Management of Resource at the Network Edge for Federated Learning
• Multi-task Federated Learning for Heterogeneous Pancreas Segmentation
• Distributed Networked Learning with Correlated Data
• LINDT: Tackling Negative Federated Learning with Local Adaptation
• Sketching Linear Classifiers over Data Streams
• A Personalized Federated Learning Algorithm: an Application in Anomaly Detection