Word Class Representations Spontaneously Emerge from Successor Representations Trained on Natural Language

Mathis Immertreu, Achim Schilling, Thomas Kinfe +1

Language models are typically trained to predict the next token in a sequence. Here, we explore an alternative predictive principle from reinforcement learning: Successor Represent…