Interleaver Design for Deep Neural Networks

Abstract

We propose a class of interleavers for a novel deep neural network (DNN) architecture that uses algorithmically pre-determined, structured sparsity to significantly lower memory and computational requirements, and speed up training. The interleavers guarantee clash-free memory accesses to eliminate idle operational cycles, optimize spread and dispersion to improve network performance, and are designed to ease the complexity of memory address computations in hardware. We present a design algorithm with mathematical proofs for these properties. We also explore interleaver variations and analyze the behavior of neural networks as a function of interleaver metrics.

Slightly abridged version presented at the 2017 51st Asilomar Conference on Signals, Systems, and Computers, copyright IEEE

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Compressing Neural Networks with the Hashing Trick
• Deep Adaptive Network: An Efficient Deep Neural Network with Sparse Binary Connections

Cited by in corpus (4)

• Pre-Defined Sparse Neural Networks with Hardware Acceleration
• Morse Code Datasets for Machine Learning
• A Highly Parallel FPGA Implementation of Sparse Neural Network Training
• Characterizing Sparse Connectivity Patterns in Neural Networks