A Genetic Programming Approach to Designing Convolutional Neural Network Architectures

Abstract

The convolutional neural network (CNN), which is one of the deep learning models, has seen much success in a variety of computer vision tasks. However, designing CNN architectures still requires expert knowledge and a lot of trial and error. In this paper, we attempt to automatically construct CNN architectures for an image classification task based on Cartesian genetic programming (CGP). In our method, we adopt highly functional modules, such as convolutional blocks and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity represented by the CGP encoding method are optimized to maximize the validation accuracy. To evaluate the proposed method, we constructed a CNN architecture for the image classification task with the CIFAR-10 dataset. The experimental result shows that the proposed method can be used to automatically find the competitive CNN architecture compared with state-of-the-art models.

arXiv commentThis is the revised version of the GECCO 2017 paper. The code of our method is available at https://github.com/sg-nm/cgp-cnn

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Practical Bayesian Optimization of Machine Learning Algorithms
• Playing Atari with Deep Reinforcement Learning
• Neural Architecture Search with Reinforcement Learning
• Densely Connected Convolutional Networks
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• CMA-ES for Hyperparameter Optimization of Deep Neural Networks
• Generative NeuroEvolution for Deep Learning
• Convolution by Evolution: Differentiable Pattern Producing Networks

Cited by in corpus (18)

• SMASH: One-Shot Model Architecture Search through HyperNetworks
• AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
• Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search
• Practical Block-wise Neural Network Architecture Generation
• Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
• HARK Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning
• BlockQNN: Efficient Block-wise Neural Network Architecture Generation
• Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling
• Multi-level CNN for lung nodule classification with Gaussian Process assisted hyperparameter optimization
• Autonomous Deep Learning: A Genetic DCNN Designer for Image Classification
• Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
• ImmuNeCS: Neural Committee Search by an Artificial Immune System
• Fast-DENSER++: Evolving Fully-Trained Deep Artificial Neural Networks
• Genetic Neural Architecture Search for automatic assessment of human sperm images
• Deep Net Triage: Analyzing the Importance of Network Layers via Structural Compression
• ImmuNetNAS: An Immune-network approach for searching Convolutional Neural Network Architectures
• Multi-objective Evolutionary Approach for Efficient Kernel Size and Shape for CNN
• Parameterless Stochastic Natural Gradient Method for Discrete Optimization and its Application to Hyper-Parameter Optimization for Neural Network