Direct Federated Neural Architecture Search

Abstract

Neural Architecture Search (NAS) is a collection of methods to craft the way neural networks are built. We apply this idea to Federated Learning (FL), wherein predefined neural network models are trained on the client/device data. This approach is not optimal as the model developers can't observe the local data, and hence, are unable to build highly accurate and efficient models. NAS is promising for FL which can search for global and personalized models automatically for the non-IID data. Most NAS methods are computationally expensive and require fine-tuning after the search, making it a two-stage complex process with possible human intervention. Thus there is a need for end-to-end NAS which can run on the heterogeneous data and resource distribution typically seen in the FL scenario. In this paper, we present an effective approach for direct federated NAS which is hardware agnostic, computationally lightweight, and a one-stage method to search for ready-to-deploy neural network models. Our results show an order of magnitude reduction in resource consumption while edging out prior art in accuracy. This opens up a window of opportunity to create optimized and computationally efficient federated learning systems.

arXiv commentUnder Review

References in corpus (14)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Neural Architecture Search with Reinforcement Learning
• DARTS: Differentiable Architecture Search
• Efficient Neural Architecture Search via Parameter Sharing
• Large-Scale Evolution of Image Classifiers
• PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Efficient Architecture Search by Network Transformation
• Random Search and Reproducibility for Neural Architecture Search
• Federated Neural Architecture Search
• MANAS: Multi-Agent Neural Architecture Search
• Real-time Federated Evolutionary Neural Architecture Search
• Fast Neural Architecture Construction using EnvelopeNets