Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
ISO: An RLVR-Native Optimization Stack
Hanqing Zhu, Wenyan Cong, Zhizhou Sha +8
Reinforcement learning with verifiable rewards (RLVR) advances the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-s…
cs.LG2025
GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
Tianhao Chen, Xin Xu, Zijing Liu +12
Modern Large Language Models, such as the LLaMA, Qwen and DeepSeek series, predominantly adopt the Pre-LayerNorm (Pre-LN) Transformer architecture. While being stable during pretra…
cs.LG2025
The Curse of Depth in Large Language Models
Wenfang Sun, Xinyuan Song, Pengxiang Li +3
In this paper, we introduce the Curse of Depth, a concept that highlights, explains, and addresses the recent observation in modern Large Language Models (LLMs) where nearly half o…