QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices

Juntao Zhao, Borui Wan, Yanghua Peng +3

A number of production deep learning clusters have attempted to explore inference hardware for DNN training, at the off-peak serving hours with many inference GPUs idling. Conducti…