Warmstarting for Scaling Language Models

Neeratyoy Mallik, Maciej Janowski, Johannes Hog +4

Scaling model sizes to scale performance has worked remarkably well for the current large language models paradigm. The research and empirical findings of various scaling studies l…