Benchmarking LLM Competence on Logical Inference over Probability Operators

Nayera Hasan, Jack Greff, Alvin Grissom

The paper presents a benchmark for testing large language models' ability to reason logically about probability expressions in English, and evaluates 29 models, revealing widesprea…