Humans and transformer LMs: Abstraction drives language learning

Jasper Jian, Christopher D. Manning

Categorization is a core component of human linguistic competence. We investigate how a transformer-based language model (LM) learns linguistic categories by comparing its behaviou…