Autonomous Deep Learning: A Genetic DCNN Designer for Image Classification

Abstract

Recent years have witnessed the breakthrough success of deep convolutional neural networks (DCNNs) in image classification and other vision applications. Although freeing users from the troublesome handcrafted feature extraction by providing a uniform feature extraction-classification framework, DCNNs still require a handcrafted design of their architectures. In this paper, we propose the genetic DCNN designer, an autonomous learning algorithm can generate a DCNN architecture automatically based on the data available for a specific image classification problem. We first partition a DCNN into multiple stacked meta convolutional blocks and fully connected blocks, each containing the operations of convolution, pooling, fully connection, batch normalization, activation and drop out, and thus convert the architecture into an integer vector. Then, we use refined evolutionary operations, including selection, mutation and crossover to evolve a population of DCNN architectures. Our results on the MNIST, Fashion-MNIST, EMNISTDigit, EMNIST-Letter, CIFAR10 and CIFAR100 datasets suggest that the proposed genetic DCNN designer is able to produce automatically DCNN architectures, whose performance is comparable to, if not better than, that of stateof- the-art DCNN models

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• ADADELTA: An Adaptive Learning Rate Method
• Deep Residual Learning for Image Recognition
• Neural Architecture Search with Reinforcement Learning
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Densely Connected Convolutional Networks
• Going Deeper with Convolutions
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• EMNIST: an extension of MNIST to handwritten letters
• Dynamic Routing Between Capsules
• A Genetic Programming Approach to Designing Convolutional Neural Network Architectures
• Zero-bias autoencoders and the benefits of co-adapting features
• Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing
• Neural Network Architecture Optimization through Submodularity and Supermodularity