A Novel Framework for Neural Architecture Search in the Hill Climbing Domain

Abstract

Neural networks have now long been used for solving complex problems of image domain, yet designing the same needs manual expertise. Furthermore, techniques for automatically generating a suitable deep learning architecture for a given dataset have frequently made use of reinforcement learning and evolutionary methods which take extensive computational resources and time. We propose a new framework for neural architecture search based on a hill-climbing procedure using morphism operators that makes use of a novel gradient update scheme. The update is based on the aging of neural network layers and results in the reduction in the overall training time. This technique can search in a broader search space which subsequently yields competitive results. We achieve a 4.96% error rate on the CIFAR-10 dataset in 19.4 hours of a single GPU training.

8 pages, 6 figures

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• Neural Architecture Search with Reinforcement Learning
• Efficient Architecture Search by Network Transformation
• Shake-Shake regularization
• Neural Optimizer Search with Reinforcement Learning
• Simple And Efficient Architecture Search for Convolutional Neural Networks