Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Learning Dynamics in Continual Pre-Training for Large Language Models
Xingjin Wang, Howe Tissue, Lu Wang +2
Continual Pre-Training (CPT) has become a popular and effective method to apply strong foundation models to specific downstream tasks. In this work, we explore the learning dynamic…
cs.CL2025
Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training
Mozhi Zhang, Howe Tissue, Lu Wang +1
We introduce~\textsc{Domain2Vec}, a novel approach that decomposes any dataset into a linear combination of several \emph{meta-domains}, a new concept designed to capture the key u…