A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutions

Laurène Vaugrante, Mathias Niepert, Thilo Hagendorff

In an era where large language models (LLMs) are increasingly integrated into a wide range of everyday applications, research into these models' behavior has surged. However, due t…