298 citations · 298 across the 10 of their papers we have counts for
3 papers · 1 filter
Data Schedules and Learning Rate Schedules Should Be Designed Together
Haodong Wen, Kairong Luo, Tingkai Yan +2
Mid-training (MT) is a training phase in LLM training pipelines that continues pretraining (PT) on smaller datasets of higher value. Taking the PT and MT phases as a whole, trainin…
Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090
Kairong Luo, Jiarui Cui, Yaorui Yin +8
Language model pretraining has become almost synonymous with prohibitive cost, placing it out of reach for much of the academic and open-source communities. Although strong open-so…
Mat2Boundary: Treating User-Defined Boundary Condition as SpMV for Distributed PDE Solvers on Block-Structured Grids
Yanzheng Cai, Mingzhe Zhang, Shengqi Chen +2
Boundary-condition (BC) handling is a major source of complexity in PDE solvers on structured and block-structured grids, especially for high-order methods and distributed-memory e…