From Federated Learning to Federated Neural Architecture Search: A Survey
arXiv:2009.05868
Abstract
Federated learning is a recently proposed distributed machine learning paradigm for privacy preservation, which has found a wide range of applications where data privacy is of primary concern. Meanwhile, neural architecture search has become very popular in deep learning for automatically tuning the architecture and hyperparameters of deep neural networks. While both federated learning and neural architecture search are faced with many open challenges, searching for optimized neural architectures in the federated learning framework is particularly demanding. This survey paper starts with a brief introduction to federated learning, including both horizontal, vertical, and hybrid federated learning. Then, neural architecture search approaches based on reinforcement learning, evolutionary algorithms and gradient-based are presented. This is followed by a description of federated neural architecture search that has recently been proposed, which is categorized into online and offline implementations, and single- and multi-objective search approaches. Finally, remaining open research questions are outlined and promising research topics are suggested.
References in corpus (38)
- Distilling the Knowledge in a Neural Network
- Deep Learning with Differential Privacy
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Neural Architecture Search with Reinforcement Learning
- Federated Learning: Strategies for Improving Communication Efficiency
- Federated Learning with Non-IID Data
- Neural Architecture Search: A Survey
- Automatically designing CNN architectures using genetic algorithm for image classification
- Robust Aggregation for Federated Learning
- Secure Federated Transfer Learning
- Efficient Neural Architecture Search via Parameter Sharing
- Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
- The Hidden Vulnerability of Distributed Learning in Byzantium
- Large-Scale Evolution of Image Classifiers
- Ensemble Distillation for Robust Model Fusion in Federated Learning
- Differentially Private Federated Learning: A Client Level Perspective
- Federated Optimization in Heterogeneous Networks
- Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
- Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
- PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
- Agnostic Federated Learning
- Single Path One-Shot Neural Architecture Search with Uniform Sampling
- One-Shot Neural Architecture Search via Self-Evaluated Template Network
- Inverting Gradients -- How easy is it to break privacy in federated learning?
- Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
- Freeze-Thaw Bayesian Optimization
- Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
- Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
- Multi-Participant Multi-Class Vertical Federated Learning
- MaskConnect: Connectivity Learning by Gradient Descent
- Asymmetrical Vertical Federated Learning
- Federated Continual Learning with Weighted Inter-client Transfer
- Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search
- Federated Neural Architecture Search
- Finding Better Topologies for Deep Convolutional Neural Networks by Evolution
- Feature-Distributed SVRG for High-Dimensional Linear Classification
- Multi-Objective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification
- Real-time Federated Evolutionary Neural Architecture Search