MoR: Mixture Of Representations For Mixed-Precision Training

Bor-Yiing Su, Peter Dykas, Mike Chrzanowski +1

Mixed-precision training is a crucial technique for scaling deep learning models, but successful mixedprecision training requires identifying and applying the right combination of…