Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds

Victoria Basmov, Yoav Goldberg, Reut Tsarfaty

We evaluate LLMs' language understanding capacities on simple inference tasks that most humans find trivial. Specifically, we target (i) grammatically-specified entailments, (ii) p…