GNMR: Runtime Stability Control for Low-Precision Large Language Model Training

Boao Kong, Weichen Jia, Engao Zhang +6

Training stability is a key bottleneck in low-precision language model training: efficient low-cost paths can still produce short-lived numerical risks at a small set of operators.…