data organization 1efficient training 1exposure balancing 1knn batching 1large language model fine-tuning 1
From the 1 of 2 linked papers with an AI index.
2 papers
cs.CL2026
Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining
Hai Wang, Chenhao Wang, Qifeng Cai +6
Large-scale pretraining corpora contain substantial duplicate content. Although document-level deduplication is widely used, removing subdocument-level redundancy remains challengi…
cs.LG2026
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
Jinliang Gao, Ning Yang, Hai Wang +2
The paper introduces SDO, a plug‑and‑play framework that dynamically organizes training data for large language model post‑training by using exposure‑driven feedback and K‑nearest‑…