Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events

James A. Michaelov, Reeka Estacio, Zhien Zhang +1

Can language models reliably predict that possible events are more likely than merely improbable ones? By teasing apart possibility, typicality, and contextual relatedness, we show…