The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason

Shanchao Liang, Spandan Garg, Roshanak Zilouchian Moghaddam

As large language models (LLMs) become increasingly capable and widely adopted, benchmarks play a central role in assessing their practical utility. For example, SWE-Bench Verified…