Real-time Federated Evolutionary Neural Architecture Search

Abstract

Federated learning is a distributed machine learning approach to privacy preservation and two major technical challenges prevent a wider application of federated learning. One is that federated learning raises high demands on communication, since a large number of model parameters must be transmitted between the server and the clients. The other challenge is that training large machine learning models such as deep neural networks in federated learning requires a large amount of computational resources, which may be unrealistic for edge devices such as mobile phones. The problem becomes worse when deep neural architecture search is to be carried out in federated learning. To address the above challenges, we propose an evolutionary approach to real-time federated neural architecture search that not only optimize the model performance but also reduces the local payload. During the search, a double-sampling technique is introduced, in which for each individual, a randomly sampled sub-model of a master model is transmitted to a number of randomly sampled clients for training without reinitialization. This way, we effectively reduce computational and communication costs required for evolutionary optimization and avoid big performance fluctuations of the local models, making the proposed framework well suited for real-time federated neural architecture search.

References in corpus (11)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Neural Architecture Search with Reinforcement Learning
• Federated Learning: Strategies for Improving Communication Efficiency
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• DARTS: Differentiable Architecture Search
• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Large-Scale Evolution of Image Classifiers
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements

Cited by in corpus (7)

• AutoML: A Survey of the State-of-the-Art
• A Survey on Evolutionary Neural Architecture Search
• Self-supervised Cross-silo Federated Neural Architecture Search
• From Federated Learning to Federated Neural Architecture Search: A Survey
• Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning
• Direct Federated Neural Architecture Search
• Federated Whole Prostate Segmentation in MRI with Personalized Neural Architectures