Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

Lisa Bylinina

A language model normally begins training with random word embeddings: whatever 'banana' means must be learned from training corpora. I implement St. Augustine's picture of word le…