Deep Learning Training on the Edge with Low-Precision Posits
arXiv:1907.13216
Abstract
Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and Fashion MNIST corpuses, where 16-bit posits outperform 16-bit floating point for end-to-end DNN training.
References in corpus (11)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Mixed Precision Training
- Training Deep Neural Networks with 8-bit Floating Point Numbers
- Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
- Rethinking floating point for deep learning
- Revisiting the Importance of Individual Units in CNNs via Ablation
- A Study of BFLOAT16 for Deep Learning Training
- Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks
- MEC: Memory-efficient Convolution for Deep Neural Network
- Mixed Precision Training With 8-bit Floating Point
- Template-Based Posit Multiplication for Training and Inferring in Neural Networks