LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets

Abstract

Federated learning is a popular distributed machine learning paradigm with enhanced privacy. Its primary goal is learning a global model that offers good performance for the participants as many as possible. The technology is rapidly advancing with many unsolved challenges, among which statistical heterogeneity (i.e., non-IID) and communication efficiency are two critical ones that hinder the development of federated learning. In this work, we propose LotteryFL -- a personalized and communication-efficient federated learning framework via exploiting the Lottery Ticket hypothesis. In LotteryFL, each client learns a lottery ticket network (i.e., a subnetwork of the base model) by applying the Lottery Ticket hypothesis, and only these lottery networks will be communicated between the server and clients. Rather than learning a shared global model in classic federated learning, each client learns a personalized model via LotteryFL; the communication cost can be significantly reduced due to the compact size of lottery networks. To support the training and evaluation of our framework, we construct non-IID datasets based on MNIST, CIFAR-10 and EMNIST by taking feature distribution skew, label distribution skew and quantity skew into consideration. Experiments on these non-IID datasets demonstrate that LotteryFL significantly outperforms existing solutions in terms of personalization and communication cost.

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Federated Learning for Mobile Keyboard Prediction
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• iDLG: Improved Deep Leakage from Gradients
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Three Approaches for Personalization with Applications to Federated Learning
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs

Cited by in corpus (10)

• Adaptive Personalized Federated Learning
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
• SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
• FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update
• FedCom: A Byzantine-Robust Local Model Aggregation Rule Using Data Commitment for Federated Learning
• Backdoor Attacks on Federated Learning with Lottery Ticket Hypothesis
• New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning
• More Industry-friendly: Federated Learning with High Efficient Design
• Constrained Differentially Private Federated Learning for Low-bandwidth Devices
• AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning