Showing cs.AIShow all
2 papers · 1 filter
cs.AI2026
How to Set the Batch Size for Large-Scale Pre-training?
Yunhua Zhou, Junhao Huang, Shuhao Xing +4
The concept of Critical Batch Size, as pioneered by OpenAI, has long served as a foundational principle for large-scale pre-training. However, with the paradigm shift towards the W…
cs.AI2026
How to Set the Learning Rate for Large-Scale Pre-training?
Yunhua Zhou, Shuhao Xing, Junhao Huang +2
Optimal configuration of the learning rate (LR) is a fundamental yet formidable challenge in large-scale pre-training. Given the stringent trade-off between training costs and mode…