Overcoming Oscillations in Quantization-Aware Training

Markus Nagel, Marios Fournarakis, Yelysei Bondarenko +1

When training neural networks with simulated quantization, we observe that quantized weights can, rather unexpectedly, oscillate between two grid-points. The importance of this eff…