Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models

Aaron Mueller, Yu Xia, Tal Linzen

Structural probing work has found evidence for latent syntactic information in pre-trained language models. However, much of this analysis has focused on monolingual models, and an…