FedMix: Approximation of Mixup under Mean Augmented Federated Learning

Abstract

Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device, thus preserving privacy and eliminating the need to store data globally. While there are promising results under the assumption of independent and identically distributed (iid) local data, current state-of-the-art algorithms suffer from performance degradation as the heterogeneity of local data across clients increases. To resolve this issue, we propose a simple framework, Mean Augmented Federated Learning (MAFL), where clients send and receive averaged local data, subject to the privacy requirements of target applications. Under our framework, we propose a new augmentation algorithm, named FedMix, which is inspired by a phenomenal yet simple data augmentation method, Mixup, but does not require local raw data to be directly shared among devices. Our method shows greatly improved performance in the standard benchmark datasets of FL, under highly non-iid federated settings, compared to conventional algorithms.

References in corpus (10)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning with Non-IID Data
• Learning Differentially Private Recurrent Language Models
• Manifold Mixup: Better Representations by Interpolating Hidden States
• EMNIST: an extension of MNIST to handwritten letters
• LEAF: A Benchmark for Federated Settings
• Bayesian Nonparametric Federated Learning of Neural Networks
• Augmenting Data with Mixup for Sentence Classification: An Empirical Study
• Federated Learning with Matched Averaging

Cited by in corpus (11)

• An overview of mixing augmentation methods and augmentation strategies
• SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
• One-shot Federated Learning without Server-side Training
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
• Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective
• Federated Learning on Non-IID Data: A Survey
• Learning Federated Representations and Recommendations with Limited Negatives
• Synthetic data shuffling accelerates the convergence of federated learning under data heterogeneity
• Dynamic Attention-based Communication-Efficient Federated Learning
• Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization
• Vicinal Feature Statistics Augmentation for Federated 3D Medical Volume Segmentation