Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search

Abstract

Federated Learning (FL) has been proved to be an effective learning framework when data cannot be centralized due to privacy, communication costs, and regulatory restrictions. When training deep learning models under an FL setting, people employ the predefined model architecture discovered in the centralized environment. However, this predefined architecture may not be the optimal choice because it may not fit data with non-identical and independent distribution (non-IID). Thus, we advocate automating federated learning (AutoFL) to improve model accuracy and reduce the manual design effort. We specifically study AutoFL via Neural Architecture Search (NAS), which can automate the design process. We propose a Federated NAS (FedNAS) algorithm to help scattered workers collaboratively searching for a better architecture with higher accuracy. We also build a system based on FedNAS. Our experiments on non-IID dataset show that the architecture searched by FedNAS can outperform the manually predefined architecture.

accepted to CVPR 2020 workshop on neural architecture search and beyond for representation learning. Code is released at https://fedml.ai

References in corpus (3)

• Towards Federated Learning at Scale: System Design
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Central Server Free Federated Learning over Single-sided Trust Social Networks

Cited by in corpus (14)

• FedML: A Research Library and Benchmark for Federated Machine Learning
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• A Field Guide to Federated Optimization
• FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks
• Federated Mutual Learning
• Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Federated Neural Architecture Search
• Self-supervised Cross-silo Federated Neural Architecture Search
• Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing
• From Federated Learning to Federated Neural Architecture Search: A Survey
• AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning
• Federated Whole Prostate Segmentation in MRI with Personalized Neural Architectures
• FDNAS: Improving Data Privacy and Model Diversity in AutoML