Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Training Dynamics of a 1.7B LLaMa Model: A Data-Efficient Approach
Miles Q. Li, Benjamin C. M. Fung, Shih-Chia Huang
Pretraining large language models is a complex endeavor influenced by multiple factors, including model architecture, data quality, training continuity, and hardware constraints. I…
cs.CL2024
On the Effectiveness of Incremental Training of Large Language Models
Miles Q. Li, Benjamin C. M. Fung, Shih-Chia Huang
Training large language models is a computationally intensive process that often requires substantial resources to achieve state-of-the-art results. Incremental layer-wise training…