53 citations · 142 across the 6 of their papers we have counts for
1 paper · 2 filters
Kaiyue Wen, Zhiyuan Li, Jason Wang +3
Training language models currently requires pre-determining a fixed compute budget because the typical cosine learning rate schedule depends on the total number of steps. In contra…