One-Shot Federated Learning

Abstract

We present one-shot federated learning, where a central server learns a global model over a network of federated devices in a single round of communication. Our approach - drawing on ensemble learning and knowledge aggregation - achieves an average relative gain of 51.5% in AUC over local baselines and comes within 90.1% of the (unattainable) global ideal. We discuss these methods and identify several promising directions of future work.

5 pages, 3 figures, 1 table. 2nd Workshop on Machine Learning on the Phone and other Consumer Devices, NeurIPs 2018

References in corpus (3)

• Distilling the Knowledge in a Neural Network
• Federated Learning: Strategies for Improving Communication Efficiency
• LEAF: A Benchmark for Federated Settings

Cited by in corpus (39)

• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Personalized Federated Learning: A Meta-Learning Approach
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• Personalized Federated Learning with Moreau Envelopes
• On the Convergence of Local Descent Methods in Federated Learning
• Federated Learning: Opportunities and Challenges
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Distilled One-Shot Federated Learning
• Salvaging Federated Learning by Local Adaptation
• Federated Learning for Healthcare Informatics
• Communication Efficiency in Federated Learning: Achievements and Challenges
• Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources
• Federated Learning Meets Natural Language Processing: A Survey
• One-shot Federated Learning without Server-side Training
• Deep Gamblers: Learning to Abstain with Portfolio Theory
• FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
• Federated Few-Shot Learning for Mobile NLP
• Practical Vertical Federated Learning with Unsupervised Representation Learning
• Parameterized Knowledge Transfer for Personalized Federated Learning
• A Federated Learning Approach for Mobile Packet Classification
• SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling
• Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
• FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
• Communication-Efficient Federated Distillation
• Federated Optimization of Smooth Loss Functions
• Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan
• Practical One-Shot Federated Learning for Cross-Silo Setting
• A Robust Federated Learning Approach for Combating Attacks Against IoT Systems Under non-IID Challenges
• Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
• Edge Bias in Federated Learning and its Solution by Buffered Knowledge Distillation
• FLAGS Framework for Comparative Analysis of Federated Learning Algorithms
• A Privacy-Preserving Federated Learning Approach for Kernel methods
• Federated Learning for Open Banking
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Representation Consolidation for Training Expert Students
• Federated Few-Shot Learning with Adversarial Learning
• FedDQ: Communication-Efficient Federated Learning with Descending Quantization
• FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning