Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Towards joint scaling laws with optimal batch size schedules
Jiaxiang Li, Zhiqi Bu, Shiyun Xu
Modern deep learning typically keeps the batch size static throughout training, thus overlooking the joint effect of learning rate and batch size on the training dynamics. In this…
cs.LG2026
Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate
Zhiqi Bu, Shiyun Xu, Jialin Mao
Deep learning has non-convex loss landscape and its optimization dynamics is hard to analyze or control. Nevertheless, the dynamics can be empirically convex-like across various ta…