Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS

Abstract

In this paper, we compare the three most popular algorithms for hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm) and attempt to use them for neural architecture search (NAS). We use these algorithms for building a convolutional neural network (search architecture). Experimental results on CIFAR-10 dataset further demonstrate the performance difference between compared algorithms. The comparison results are based on the execution time of the above algorithms and accuracy of the proposed models.

11 pages, 5 figures, 3 tables

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Neural Architecture Search: A Survey
• AdaNet: A Scalable and Flexible Framework for Automatically Learning Ensembles

Cited by in corpus (10)

• A Survey on Evolutionary Neural Architecture Search
• Utilising Deep Learning Techniques for Effective Zero-Day Attack Detection
• Multi-Objective Loss Balancing for Physics-Informed Deep Learning
• Utilizing XAI technique to improve autoencoder based model for computer network anomaly detection with shapley additive explanation(SHAP)
• Survival Prediction of Children Undergoing Hematopoietic Stem Cell Transplantation Using Different Machine Learning Classifiers by Performing Chi-squared Test and Hyper-parameter Optimization: A Retrospective Analysis
• GNN-Geo: A Graph Neural Network-based Fine-grained IP geolocation Framework
• Massive MIMO CSI Feedback using Channel Prediction: How to Avoid Machine Learning at UE?
• Attention to the strengths of physical interactions: Transformer and graph-based event classification for particle physics experiments
• SincVAE: A new semi-supervised approach to improve anomaly detection on EEG data using SincNet and variational autoencoder
• Compensating Removed Frequency Components: Thwarting Voice Spectrum Reduction Attacks