Neural Architecture Search: Insights from 1000 Papers

Abstract

In the past decade, advances in deep learning have resulted in breakthroughs in a variety of areas, including computer vision, natural language understanding, speech recognition, and reinforcement learning. Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas. Neural architecture search (NAS), the process of automating the design of neural architectures for a given task, is an inevitable next step in automating machine learning and has already outpaced the best human-designed architectures on many tasks. In the past few years, research in NAS has been progressing rapidly, with over 1000 papers released since 2020 (Deng and Lindauer, 2021). In this survey, we provide an organized and comprehensive guide to neural architecture search. We give a taxonomy of search spaces, algorithms, and speedup techniques, and we discuss resources such as benchmarks, best practices, other surveys, and open-source libraries.

Cited by in corpus (5)

• DAS-N2N: Machine learning Distributed Acoustic Sensing (DAS) signal denoising without clean data
• Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters
• Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search