6 papers
DGA-Muon: Decoupled Geometry-Aligned Adaptive Scaling for Muon
Wenpeng Zhang, Runsheng Yu
While NorMuon has achieved strong empirical performance in pretraining, its underlying adaptive mechanism remains largely heuristic and poorly understood. In this work, we provide…
Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods
Wenpeng Zhang, Runsheng Yu, Peilin Zhao
Adaptive optimization methods such as AdaGrad and Adam are widely used in modern deep neural network training, but their adaptive scaling is primarily designed for vector-valued pa…
How Much Information Can a Vision Token Hold? A Scaling Law for Recognition Limits in VLMs
Shuxin Zhuang, Zi Liang, Runsheng Yu +4
Recent vision-centric approaches have made significant strides in long-context modeling. Represented by DeepSeek-OCR, these models encode rendered text into continuous vision token…
Particle Dynamics for Latent-Variable Energy-Based Models
Shiqin Tang, Shuxin Zhuang, Rong Feng +3
Latent-variable energy-based models (LVEBMs) assign a single normalized energy to joint pairs of observed data and latent variables, offering expressive generative modeling while c…
Tailed Low-Rank Matrix Factorization for Similarity Matrix Completion
Changyi Ma, Runsheng Yu, Xiao Chen +1
Similarity matrix serves as a fundamental tool at the core of numerous downstream machine-learning tasks. However, missing data is inevitable and often results in an inaccurate sim…
Direct Alignment of Language Models via Quality-Aware Self-Refinement
Runsheng Yu, Yong Wang, Xiaoqi Jiao +2
Reinforcement Learning from Human Feedback (RLHF) has been commonly used to align the behaviors of Large Language Models (LLMs) with human preferences. Recently, a popular alternat…