3 papers
cs.AI2026
CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data
Peiguang Li, Yongwei Zhou, Juncheng Diao +12
Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance. Ho…
cs.AI2026
HIPIF: Hierarchical Planning and Information Folding for Long-Horizon LLM Agent Learning
Juncheng Diao, Zhicong Lu, Peiguang Li +6
While Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents across a wide range of tasks, their performance often degrades in multi-turn long-hori…
cs.CL2026
Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training
Yongwei Zhou, Juncheng Diao, Junlin Shang +2
The efficacy of continued pre-training for Large Language Models (LLMs) hinges upon hyperparameter configurations, such as learning rate and batch size. However, current practices…