Federated Learning with Personalization Layers

Abstract

The emerging paradigm of federated learning strives to enable collaborative training of machine learning models on the network edge without centrally aggregating raw data and hence, improving data privacy. This sharply deviates from traditional machine learning and necessitates the design of algorithms robust to various sources of heterogeneity. Specifically, statistical heterogeneity of data across user devices can severely degrade the performance of standard federated averaging for traditional machine learning applications like personalization with deep learning. This paper pro-posesFedPer, a base + personalization layer approach for federated training of deep feedforward neural networks, which can combat the ill-effects of statistical heterogeneity. We demonstrate effectiveness ofFedPerfor non-identical data partitions ofCIFARdatasetsand on a personalized image aesthetics dataset from Flickr.

Cited by in corpus (19)

• A Field Guide to Federated Optimization
• A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
• A distillation-based approach integrating continual learning and federated learning for pervasive services
• Parameterized Knowledge Transfer for Personalized Federated Learning
• Evaluation and comparison of federated learning algorithms for Human Activity Recognition on smartphones
• Personalized Federated Learning with Gaussian Processes
• Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes
• Federated Learning on Non-IID Data: A Survey
• Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models
• Federated Learning and catastrophic forgetting in pervasive computing: demonstration in HAR domain
• FedHealth 2: Weighted Federated Transfer Learning via Batch Normalization for Personalized Healthcare
• IFedAvg: Interpretable Data-Interoperability for Federated Learning
• Modular Federated Learning
• Federated Learning for Open Banking
• Demystifying the Effects of Non-Independence in Federated Learning
• Canoe : A System for Collaborative Learning for Neural Nets
• A Personalized Federated Learning Algorithm: an Application in Anomaly Detection
• CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning