Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks

Charbel Sakr, Naigang Wang, Chia-Yu Chen +4

Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-pr…