Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats

Brian Chmiel, Ron Banner, Elad Hoffer +2

Quantization of the weights and activations is one of the main methods to reduce the computational footprint of Deep Neural Networks (DNNs) training. Current methods enable 4-bit q…