Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networks

Léopold Cambier, Anahita Bhiwandiwalla, Ting Gong +3

Training with larger number of parameters while keeping fast iterations is an increasingly adopted strategy and trend for developing better performing Deep Neural Network (DNN) mod…