2 papers
cs.CL2024
The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance
Kyle Moore, Jesse Roberts, Thao Pham +2
Cloze testing is a common method for measuring the behavior of large language models on a number of benchmark tasks. Using the MMLU dataset, we show that the base-rate probability…
cs.CL2024
Large Language Model Recall Uncertainty is Modulated by the Fan Effect
Jesse Roberts, Kyle Moore, Thao Pham +2
This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual…