Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models

Michael Hanna, Aaron Mueller

Autoregressive transformer language models (LMs) possess strong syntactic abilities, often successfully handling phenomena from agreement to NPI licensing. However, the features th…