Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors
Alexander Hägele, Alejandro Hernández-Cano, Atli Kosson +1
Modern neural network training relies on optimizers such as Adam and Muon which act on each weight matrix as a single object. Yet every weight matrix carries two distinct quantitie…
cs.LG2025
Towards Fully FP8 GEMM LLM Training at Scale
Alejandro Hernández-Cano, Dhia Garbaya, Imanol Schlag +1
Despite the significant potential of FP8 data formats for large language model (LLM) pre-training, their adoption has been limited due to challenges in maintaining stability at sca…