Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

Vijeta Deshpande, Dan Pechi, Shree Thatte +2

In recent years, language models have drastically grown in size, and the abilities of these models have been shown to improve with scale. The majority of recent scaling laws studie…