Showing 2026Show all
3 papers · 1 filter
cs.AI2026
DGA-Muon: Decoupled Geometry-Aligned Adaptive Scaling for Muon
Wenpeng Zhang, Runsheng Yu
While NorMuon has achieved strong empirical performance in pretraining, its underlying adaptive mechanism remains largely heuristic and poorly understood. In this work, we provide…
cs.LG2026
Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods
Wenpeng Zhang, Runsheng Yu, Peilin Zhao
Adaptive optimization methods such as AdaGrad and Adam are widely used in modern deep neural network training, but their adaptive scaling is primarily designed for vector-valued pa…
cs.LG2026
How Much Information Can a Vision Token Hold? A Scaling Law for Recognition Limits in VLMs
Shuxin Zhuang, Zi Liang, Runsheng Yu +4
Recent vision-centric approaches have made significant strides in long-context modeling. Represented by DeepSeek-OCR, these models encode rendered text into continuous vision token…