FedSC: Federated Learning with Semantic-Aware Collaboration

Abstract

Federated learning (FL) aims to train models collaboratively across clients without sharing data for privacy-preserving. However, one major challenge is the data heterogeneity issue, which refers to the biased labeling preferences at multiple clients. A number of existing FL methods attempt to tackle data heterogeneity locally (e.g., regularizing local models) or globally (e.g., fine-tuning global model), often neglecting inherent semantic information contained in each client. To explore the possibility of using intra-client semantically meaningful knowledge in handling data heterogeneity, in this paper, we propose Federated Learning with Semantic-Aware Collaboration (FedSC) to capture client-specific and class-relevant knowledge across heterogeneous clients. The core idea of FedSC is to construct relational prototypes and consistent prototypes at semantic-level, aiming to provide fruitful class underlying knowledge and stable convergence signals in a prototype-wise collaborative way. On the one hand, FedSC introduces an inter-contrastive learning strategy to bring instance-level embeddings closer to relational prototypes with the same semantics and away from distinct classes. On the other hand, FedSC devises consistent prototypes via a discrepancy aggregation manner, as a regularization penalty to constrain the optimization region of the local model. Moreover, a theoretical analysis for FedSC is provided to ensure a convergence guarantee. Experimental results on various challenging scenarios demonstrate the effectiveness of FedSC and the efficiency of crucial components.

12 pages, KDD 2025

References in corpus (31)

• A Simple Framework for Contrastive Learning of Visual Representations
• Prototypical Networks for Few-shot Learning
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Challenges, Methods, and Future Directions
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• Momentum Contrast for Unsupervised Visual Representation Learning
• Contrastive Representation Learning: A Framework and Review
• Visualizing the Loss Landscape of Neural Nets
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• TADAM: Task dependent adaptive metric for improved few-shot learning
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Federated Learning from Pre-Trained Models: A Contrastive Learning Approach
• Data-Free Knowledge Distillation for Heterogeneous Federated Learning
• Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning
• Preservation of the Global Knowledge by Not-True Distillation in Federated Learning
• Federated Learning with Buffered Asynchronous Aggregation
• On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond
• Robust Classification with Convolutional Prototype Learning
• FedDisco: Federated Learning with Discrepancy-Aware Collaboration
• FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction
• Adaptive Prototype Learning and Allocation for Few-Shot Segmentation
• Text-Adaptive Multiple Visual Prototype Matching for Video-Text Retrieval
• UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
• A Closer Look at Prototype Classifier for Few-shot Image Classification
• Towards Fair Federated Learning with Zero-Shot Data Augmentation
• Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection
• Online Prototype Learning for Online Continual Learning
• Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching
• FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation