The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation

Ian Magnusson, Noah A. Smith, Pang Wei Koh

The scaling relationship of compute and performance for language models (LMs) has been extensively studied; more recently, models are being designed to scale memory use, as well. I…