2 papers
cs.LG2026
SPARKLING: Balancing Signal Preservation and Symmetry Breaking for Width-Progressive Learning
Qifan Yu, Xinyu Ma, Zhijian Zhuo +7
Progressive Learning (PL) reduces pre-training computational overhead by gradually increasing model scale. While prior work has extensively explored depth expansion, width expansio…
cs.CL2025
Model Merging in Pre-training of Large Language Models
Yunshui Li, Yiyuan Ma, Shen Yan +23
Model merging has emerged as a promising technique for enhancing large language models, though its application in large-scale pre-training remains relatively unexplored. In this pa…