FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

Abstract

As a promising distributed machine learning paradigm, Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. By adopting the classic one-to-multi training scheme (i.e., FedAvg), where the cloud server dispatches one single global model to multiple involved clients, conventional FL methods can achieve collaborative model training without data sharing. However, since only one global model cannot always accommodate all the incompatible convergence directions of local models, existing FL approaches greatly suffer from inferior classification accuracy. To address this issue, we present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. Unlike traditional FL methods, in each round of FL training, FedCross uses multiple middleware models to conduct weighted fusion individually. Since the middleware models used by FedCross can quickly converge into the same flat valley in terms of loss landscapes, the generated global model can achieve a well-generalization. Experimental results on various well-known datasets show that, compared with state-of-the-art FL methods, FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• Performance Optimization for Federated Person Re-identification via Benchmark Analysis
• FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling
• GitFL: Adaptive Asynchronous Federated Learning using Version Control
• AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systems
• Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning
• AdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systems

Cited by in corpus (1)

• Gradients as an Action: Towards Communication-Efficient Federated Recommender Systems via Adaptive Action Sharing