API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces

Jennifer Wang, Joachim Baumann, Daniel E. Ho +1

Benchmark scores are a central currency in model releases: they inform purchasing decisions, shape public trust, and influence policy. Yet, a key assumption underlying benchmark sc…