FedALA: Adaptive Local Aggregation for Personalized Federated Learning

Abstract

A key challenge in federated learning (FL) is the statistical heterogeneity that impairs the generalization of the global model on each client. To address this, we propose a method Federated learning with Adaptive Local Aggregation (FedALA) by capturing the desired information in the global model for client models in personalized FL. The key component of FedALA is an Adaptive Local Aggregation (ALA) module, which can adaptively aggregate the downloaded global model and local model towards the local objective on each client to initialize the local model before training in each iteration. To evaluate the effectiveness of FedALA, we conduct extensive experiments with five benchmark datasets in computer vision and natural language processing domains. FedALA outperforms eleven state-of-the-art baselines by up to 3.27% in test accuracy. Furthermore, we also apply ALA module to other federated learning methods and achieve up to 24.19% improvement in test accuracy.

arXiv commentAccepted by AAAI 2023

References in corpus (18)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Character-level Convolutional Networks for Text Classification
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Federated Learning with Non-IID Data
• Visualizing the Loss Landscape of Neural Nets
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• Ditto: Fair and Robust Federated Learning Through Personalization
• Personalized Federated Learning with Moreau Envelopes
• Adaptive Gradient Methods with Dynamic Bound of Learning Rate
• Exploiting Shared Representations for Personalized Federated Learning
• Data-Free Knowledge Distillation for Heterogeneous Federated Learning
• Personalized Federated Learning using Hypernetworks
• Personalized Federated Learning with First Order Model Optimization
• Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning

Cited by in corpus (10)

• FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
• PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
• UNIDEAL: Curriculum Knowledge Distillation Federated Learning
• Convergence-Privacy-Fairness Trade-Off in Personalized Federated Learning
• Towards Fast and Stable Federated Learning: Confronting Heterogeneity via Knowledge Anchor
• Enhancing Convergence, Privacy and Fairness for Wireless Personalized Federated Learning: Quantization-Assisted Min-Max Fair Scheduling
• FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks
• FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Data
• FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning
• Controlled privacy leakage propagation throughout overlapping grouped learning