Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives

Karolina Drożdż, Micha Heilbron

Understanding language requires tracking entities across discourse - i.e., knowing where things are and how they change, even when not explicitly stated. Whether language models pe…