Showing 2025Show all
2 papers · 1 filter
cs.LG2025
Spectral Alignment as Predictor of Loss Explosion in Neural Network Training
Haiquan Qiu, You Wu, Yingjie Tan +2
Loss explosions in training deep neural networks can nullify multi-million dollar training runs. Conventional monitoring metrics like weight and gradient norms are often lagging an…
cs.LG2025
Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
Haiquan Qiu, Quanming Yao
The pursuit of computational efficiency has driven the adoption of low-precision formats for training transformer models. However, this progress is often hindered by notorious trai…