1.1k citations · 1.2k across the 3 of their papers we have counts for
3 papers · 1 filter
Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
Charbel Sakr, Naigang Wang, Chia-Yu Chen +4
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-pr…
AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Training
Chia-Yu Chen, Jungwook Choi, Daniel Brand +3
Highly distributed training of Deep Neural Networks (DNNs) on future compute platforms (offering 100 of TeraOps/s of computational capacity) is expected to be severely communicatio…
Deep Learning with Limited Numerical Precision
Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan +1
Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computa…