Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language

Mathis Immertreu, Achim Schilling, Thomas Kinfe +1

Neural language models are typically trained on next-token prediction, although linguistic structure spans multiple temporal scales. Successor representations (SRs) make this horiz…