Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
The Stability of Singular Distribution: A Spectral Perspective on the Two-Phase Dynamics of Language Model Pre-training
Hongtao Zhang, Wenjie Zhou, Chenxi Jia +2
Large language model pre-training typically exhibits a two-phase trajectory: a fast initial loss drop followed by a prolonged slow improvement. We identify an underlying spectral p…
cs.LG2026
Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
Wenjie Zhou, Bohan Wang, Hongtao Zhang +3
Model merging has emerged as a lightweight paradigm for enhancing Large Language Models (LLMs), yet its underlying mechanisms remain poorly understood. In this work, we analyze lat…
cs.LG2026
RaPA: Enhancing Transferable Targeted Attacks via Random Parameter Pruning
Tongrui Su, Qingbin Li, Shengyu Zhu +2
Compared to untargeted attacks, targeted transfer-based attack is still suffering from much lower Attack Success Rates (ASRs), although significant improvements have been achieved…