CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models

Jörg Frohberg, Frank Binder

We introduce the CRASS (counterfactual reasoning assessment) data set and benchmark utilizing questionized counterfactual conditionals as a novel and powerful tool to evaluate larg…