FedSiam: Towards Adaptive Federated Semi-Supervised Learning

Abstract

Federated learning (FL) has emerged as an effective technique to co-training machine learning models without actually sharing data and leaking privacy. However, most existing FL methods focus on the supervised setting and ignore the utilization of unlabeled data. Although there are a few existing studies trying to incorporate unlabeled data into FL, they all fail to maintain performance guarantees or generalization ability in various real-world settings. In this paper, we focus on designing a general framework FedSiam to tackle different scenarios of federated semi-supervised learning, including four settings in the labels-at-client scenario and two setting in the labels-at-server scenario. FedSiam is built upon a siamese network into FL with a momentum update to handle the non-IID challenges introduced by unlabeled data. We further propose a new metric to measure the divergence of local model layers within the siamese network. Based on the divergence, FedSiam can automatically select layer-level parameters to be uploaded to the server in an adaptive manner. Experimental results on three datasets under two scenarios with different data distribution settings demonstrate that the proposed FedSiam framework outperforms state-of-the-art baselines.

References in corpus (19)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Bootstrap your own latent: A new approach to self-supervised Learning
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Federated Learning with Non-IID Data
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Federated Learning for Mobile Keyboard Prediction
• On the Convergence of FedAvg on Non-IID Data
• Federated Machine Learning: Concept and Applications
• Semi-Supervised Learning with Ladder Networks
• Differentially Private Federated Learning: A Client Level Perspective
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Practical Secure Aggregation for Federated Learning on User-Held Data
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
• Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
• Exploiting Unlabeled Data in Smart Cities using Federated Learning