M+Adam: Low-Precision Training via Additive-Multiplicative Optimization

Xiaoyuan Liang, Sebastian Loeschcke, Mads Toftrup +1

Training with quantized weights can reduce costs but often results in degraded accuracy, especially when optimization is carried out in low precision, without storing high-precisio…