5 citations · 13 across the 8 of their papers we have counts for
5 papers · 1 filter
Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited
Anthony G Cohn, Robert E Blackwell
We investigate the abilities of 28 Large language Models (LLMs) to reason about cardinal directions (CDs) using a benchmark generated from a set of templates, extensively testing a…
An Empirical Study of Conformal Prediction in LLM with ASP Scaffolds for Robust Reasoning
Navdeep Kaur, Lachlan McPheat, Alessandra Russo +2
In this paper, we examine the use of Conformal Language Modelling (CLM) alongside Answer Set Programming (ASP) to enhance the performance of standard open-weight LLMs on complex mu…
Can Large Language Models Reason about the Region Connection Calculus?
Anthony G Cohn, Robert E Blackwell
Qualitative Spatial Reasoning is a well explored area of Knowledge Representation and Reasoning and has multiple applications ranging from Geographical Information Systems to Robot…
Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores
Robert E. Blackwell, Jon Barry, Anthony G. Cohn
Large language models (LLMs) are stochastic, and not all models give deterministic answers, even when setting temperature to zero with a fixed random seed. However, few benchmark s…
Evaluating the Ability of Large Language Models to Reason about Cardinal Directions
Anthony G Cohn, Robert E Blackwell
We investigate the abilities of a representative set of Large language Models (LLMs) to reason about cardinal directions (CDs). To do so, we create two datasets: the first, co-crea…