2 papers
cs.LG2026
Holistic Data Scheduler for LLM Pre-training via Multi-Objective Reinforcement Learning
Chenhao Dang, Jing Ma, Mingjie Liao
The composition of training data, governed by the diversity of sources and their mixing strategy, is a cornerstone of Large Language Model (LLM) pre-training. Online Data Mixing (O…
cs.LG2026
AC-ODM: Actor--Critic Online Data Mixing for Sample-Efficient LLM Pretraining
Jing Ma, Chenhao Dang, Mingjie Liao
Optimizing pretraining data composition is pivotal for LLM generalization. While dynamic mixing outperforms static strategies by capturing evolving training dynamics, current metho…