2 papers
cs.CL2026
Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining
Hai Wang, Chenhao Wang, Qifeng Cai +6
Large-scale pretraining corpora contain substantial duplicate content. Although document-level deduplication is widely used, removing subdocument-level redundancy remains challengi…
cs.LG2026
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
Jinliang Gao, Ning Yang, Hai Wang +2
Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules. However, da…