Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Model Merging in Pre-training of Large Language Models
Yunshui Li, Yiyuan Ma, Shen Yan +23
Model merging has emerged as a promising technique for enhancing large language models, though its application in large-scale pre-training remains relatively unexplored. In this pa…
cs.CL2025
Efficient Pretraining Length Scaling
Bohong Wu, Shen Yan, Sijun Zhang +4
Recent advances in large language models have demonstrated the effectiveness of length scaling during post-training, yet its potential in pre-training remains underexplored. We pre…