Structured Convolution Matrices for Energy-efficient Deep learning
arXiv:1606.02407
Abstract
We derive a relationship between network representation in energy-efficient neuromorphic architectures and block Toplitz convolutional matrices. Inspired by this connection, we develop deep convolutional networks using a family of structured convolutional matrices and achieve state-of-the-art trade-off between energy efficiency and classification accuracy for well-known image recognition tasks. We also put forward a novel method to train binary convolutional networks by utilising an existing connection between noisy-rectified linear units and binary activations.
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Improving neural networks by preventing co-adaptation of feature detectors
- cuDNN: Efficient Primitives for Deep Learning
- Compressing Deep Convolutional Networks using Vector Quantization
- Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
- Compressing Neural Networks with the Hashing Trick
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
- Six networks on a universal neuromorphic computing substrate
- Gradient learning in spiking neural networks by dynamic perturbation of conductances
- Structured Transforms for Small-Footprint Deep Learning
- Noisy Activation Functions
- Adjustable Bounded Rectifiers: Towards Deep Binary Representations