16 citations · 37 across the 18 of their papers we have counts for
Showing 2023 · cs.CLShow all
2 papers · 2 filters
cs.CL2023★ 16 cited
Continual Pre-training of Language Models
Zixuan Ke, Yijia Shao, Haowei Lin +3
Language models (LMs) have been instrumental for the rapid advance of natural language processing. This paper studies continual pre-training of LMs, in particular, continual domain…
cs.CL2023★ 1 cited
Adapting a Language Model While Preserving its General Knowledge
Zixuan Ke, Yijia Shao, Haowei Lin +3
Domain-adaptive pre-training (or DA-training for short), also known as post-training, aims to train a pre-trained general-purpose language model (LM) using an unlabeled corpus of a…