Similar Models, Broken Odds: Where Luce's Axiom Fails in Chatbot Arena

Soovadeep Bakshi, Apurba Bose, Aishwarya Bose +1

Chatbot Arena ranks language models with the Bradley-Terry model, which gives each model one score and so assumes Luce's choice axiom. Choice theory predicts where this fails: near…