Why Global LLM Leaderboards Are Misleading: Small Portfolios for Heterogeneous Supervised ML

Jai Moondra, Ayela Chughtai, Bhargavi Lanka +1

Ranking LLMs via pairwise human feedback underpins current leaderboards for open-ended tasks, such as creative writing and problem-solving. We analyze ~89K comparisons in 116 langu…