Challenges in Trustworthy Human Evaluation of Chatbots

Wenting Zhao, Alexander M. Rush, Tanya Goyal

Open community-driven platforms like Chatbot Arena that collect user preference data from site visitors have gained a reputation as one of the most trustworthy publicly available b…