most citedD-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models

3 citations · 3 across the 2 of their papers we have counts for

collaborators

2 papers

cs.CL2024★ 3 cited

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models

Haoran Que, Jiaheng Liu, Ge Zhang +13

Continual Pre-Training (CPT) on Large Language Models (LLMs) has been widely used to expand the model's fundamental understanding of specific downstream domains (e.g., math and cod…

cs.CL2024

E^2-LLM: Efficient and Extreme Length Extension of Large Language Models

Jiaheng Liu, Zhiqi Bai, Yuanxing Zhang +11

Typically, training LLMs with long context sizes is computationally expensive, requiring extensive training hours and GPU resources. Existing long-context extension methods usually…