Personalized Cross-Silo Federated Learning on Non-IID Data

Abstract

Non-IID data present a tough challenge for federated learning. In this paper, we explore a novel idea of facilitating pairwise collaborations between clients with similar data. We propose FedAMP, a new method employing federated attentive message passing to facilitate similar clients to collaborate more. We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Our extensive experiments on benchmark data sets demonstrate the superior performance of the proposed methods.

arXiv commentAccepted by AAAI 2021. The API of this work is available at Huawei Cloud (https://developer.huaweicloud.com/develop/aigallery/notebook/detail?id=6d4a9521-6a4d-4b6d-b84d-943d7c7b1cbd), free registration is required before use

Cited by in corpus (19)

• A Field Guide to Federated Optimization
• Federated Multi-Task Learning under a Mixture of Distributions
• Federated Graph Classification over Non-IID Graphs
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning
• Parameterized Knowledge Transfer for Personalized Federated Learning
• Federated Reconstruction: Partially Local Federated Learning
• Personalized Federated Learning through Local Memorization
• Model-Contrastive Federated Learning
• Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing
• Personalized Federated Learning with Gaussian Processes
• Federated Learning on Non-IID Data: A Survey
• New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning
• Personalized Federated Learning over non-IID Data for Indoor Localization
• FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning
• ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning
• DVFL: A Vertical Federated Learning Method for Dynamic Data
• Data privacy protection in microscopic image analysis for material data mining
• Efficient Federated Learning for AIoT Applications Using Knowledge Distillation