The emergence of number and syntax units in LSTM language models

Yair Lakretz, German Kruszewski, Theo Desbordes +3

Recent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however n…