2 papers
cs.CL2024
Mini-Giants: "Small" Language Models and Open Source Win-Win
Zhengping Zhou, Lezhi Li, Xinxi Chen +1
ChatGPT is phenomenal. However, it is prohibitively expensive to train and refine such giant models. Fortunately, small language models are flourishing and becoming more and more c…
cs.CL2024
Inverse Scaling: When Bigger Isn't Better
Ian R. McKenzie, Alexander Lyzhov, Michael Pieler +24
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, w…