Tolerance Principle and Small Language Model Learning

Adam E. Friedman, Stevan Harnad, Rushen Shi

Modern language models like GPT-3, BERT, and LLaMA require massive training data, yet with sufficient training they reliably learn to distinguish grammatical from ungrammatical sen…