FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy

Abstract

Recently, personalized federated learning (pFL) has attracted increasing attention in privacy protection, collaborative learning, and tackling statistical heterogeneity among clients, e.g., hospitals, mobile smartphones, etc. Most existing pFL methods focus on exploiting the global information and personalized information in the client-level model parameters while neglecting that data is the source of these two kinds of information. To address this, we propose the Federated Conditional Policy (FedCP) method, which generates a conditional policy for each sample to separate the global information and personalized information in its features and then processes them by a global head and a personalized head, respectively. FedCP is more fine-grained to consider personalization in a sample-specific manner than existing pFL methods. Extensive experiments in computer vision and natural language processing domains show that FedCP outperforms eleven state-of-the-art methods by up to 6.69%. Furthermore, FedCP maintains its superiority when some clients accidentally drop out, which frequently happens in mobile settings. Our code is public at https://github.com/TsingZ0/FedCP.

arXiv commentAccepted by KDD 2023

References in corpus (21)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distilling the Knowledge in a Neural Network
• Federated Learning: Challenges, Methods, and Future Directions
• Categorical Reparameterization with Gumbel-Softmax
• Character-level Convolutional Networks for Text Classification
• Federated Learning for Internet of Things: A Comprehensive Survey
• Towards Personalized Federated Learning
• Federated Machine Learning: Concept and Applications
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Federated Learning with Personalization Layers
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• Convergence Analysis of Two-layer Neural Networks with ReLU Activation
• Ditto: Fair and Robust Federated Learning Through Personalization
• LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix Factorization
• On Bridging Generic and Personalized Federated Learning for Image Classification
• TLSAN: Time-aware Long- and Short-term Attention Network for Next-item Recommendation
• FedDisco: Federated Learning with Discrepancy-Aware Collaboration
• Connecting Low-Loss Subspace for Personalized Federated Learning

Cited by in corpus (4)

• Personalized Federated Continual Learning via Multi-granularity Prompt
• PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
• DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations
• Controlled privacy leakage propagation throughout overlapping grouped learning