Aligning Language Model Benchmarks with Pairwise Preferences

Marco Gutierrez, Xinyi Leng, Hannah Cyberey +3

Language model benchmarks are pervasive and computationally-efficient proxies for real-world performance. However, many recent works find that benchmarks often fail to predict real…