2 papers
cs.LG2026
Dispelling the Curse of Singularities in Neural Network Optimizations
Hengjie Cao, Mengyi Chen, Yifeng Yang +11
This work investigates the optimization instability of deep neural networks from a less-explored yet insightful perspective: the emergence and amplification of singularities in the…
cs.LG2025
Metis: Training LLMs with FP4 Quantization
Hengjie Cao, Mengyi Chen, Yifeng Yang +13
This work identifies anisotropy in the singular value spectra of parameters, activations, and gradients as the fundamental barrier to low-bit training of large language models (LLM…