Benchmark Illusion: Disagreement among LLMs and Its Scientific Consequences

Eddie Yang, Dashun Wang

Benchmarks underpin how progress in large language models (LLMs) is measured and trusted. Yet our analyses reveal that apparent convergence in benchmark accuracy can conceal deep e…