works on
data organization 1efficient training 1exposure balancing 1knn batching 1large language model fine-tuning 1

From the 1 of 2 linked papers with an AI index.

collaborators
Showing cs.CLShow all

1 paper · 1 filter

cs.CL2026

Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining

Hai Wang, Chenhao Wang, Qifeng Cai +6

Large-scale pretraining corpora contain substantial duplicate content. Although document-level deduplication is widely used, removing subdocument-level redundancy remains challengi…