No Multiplication? No Floating Point? No Problem! Training Networks for Efficient Inference
arXiv:1809.09244
Abstract
For successful deployment of deep neural networks on highly--resource-constrained devices (hearing aids, earbuds, wearables), we must simplify the types of operations and the memory/power resources used during inference. Completely avoiding inference-time floating-point operations is one of the simplest ways to design networks for these highly-constrained environments. By discretizing both our in-network non-linearities and our network weights, we can move to simple, compact networks without floating point operations, without multiplications, and avoid all non-linear function computations. Our approach allows us to explore the spectrum of possible networks, ranging from fully continuous versions down to networks with bi-level weights and activations. Our results show that discretization can be done without loss of performance and that we can train a network that will successfully operate without floating-point, without multiplication, and with less RAM on both regression tasks (auto encoding) and multi-class classification tasks (ImageNet). The memory needed to deploy our discretized networks is less than one third of the equivalent architecture that does use floating-point operations.
References in corpus (12)
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Training and Inference with Integers in Deep Neural Networks
- Loss-aware Binarization of Deep Networks
- Compression Artifacts Removal Using Convolutional Neural Networks
- Real-Time Adaptive Image Compression
- Techniques for Learning Binary Stochastic Feedforward Neural Networks
- Empirical Explorations in Training Networks with Discrete Activations