4 citations · 9 across the 11 of their papers we have counts for
3 papers · 1 filter
Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers
Lorenzo Pacchiardi, Marko Tesic, Lucy G. Cheke +1
The integrity of AI benchmarks is fundamental to accurately assess the capabilities of AI systems. The internal validity of these benchmarks - i.e., making sure they are free from…
A little less conversation, a little more action, please: Investigating the physical common-sense of LLMs in a 3D embodied environment
Matteo G. Mecattaf, Ben Slater, Marko Tešić +3
As general-purpose tools, Large Language Models (LLMs) must often reason about everyday physical environments. In a question-and-answer capacity, understanding the interactions of…
100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instances
Lorenzo Pacchiardi, Lucy G. Cheke, José Hernández-Orallo
Predicting the performance of LLMs on individual task instances is essential to ensure their reliability in high-stakes applications. To do so, a possibility is to evaluate the con…