Language model acceptability judgements are not always robust to context

Koustuv Sinha, Jon Gauthier, Aaron Mueller +4

Targeted syntactic evaluations of language models ask whether models show stable preferences for syntactically acceptable content over minimal-pair unacceptable inputs. Most target…