Evolutionary Synthesis of Deep Neural Networks via Synaptic Cluster-driven Genetic Encoding
arXiv:1609.01360
Abstract
There has been significant recent interest towards achieving highly efficient deep neural network architectures. A promising paradigm for achieving this is the concept of evolutionary deep intelligence, which attempts to mimic biological evolution processes to synthesize highly-efficient deep neural networks over successive generations. An important aspect of evolutionary deep intelligence is the genetic encoding scheme used to mimic heredity, which can have a significant impact on the quality of offspring deep neural networks. Motivated by the neurobiological phenomenon of synaptic clustering, we introduce a new genetic encoding scheme where synaptic probability is driven towards the formation of a highly sparse set of synaptic clusters. Experimental results for the task of image classification demonstrated that the synthesized offspring networks using this synaptic cluster-driven genetic encoding scheme can achieve state-of-the-art performance while having network architectures that are not only significantly more efficient (with a ~125-fold decrease in synapses for MNIST) compared to the original ancestor network, but also tailored for GPU-accelerated machine learning applications.
arXiv comment5 pages
References in corpus (8)
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- End to End Learning for Self-Driving Cars
- Compressing Deep Convolutional Networks using Vector Quantization
- MatConvNet - Convolutional Neural Networks for MATLAB
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Learning Structured Sparsity in Deep Neural Networks
- Semantic Image Segmentation via Deep Parsing Network
- Monocular Object Instance Segmentation and Depth Ordering with CNNs
Cited by in corpus (3)
- SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesis
- Exploring the Imposition of Synaptic Precision Restrictions For Evolutionary Synthesis of Deep Neural Networks
- Mitigating Architectural Mismatch During the Evolutionary Synthesis of Deep Neural Networks