1 citations · 1 across the 9 of their papers we have counts for
3 papers · 2 filters
Game Arena: Strategic LLM Evaluation in Competitive Environments
Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena…
Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
Adam Fisch, Shubhendu Trivedi, Fantine Huot +5
Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist w…
Active Evaluation of General Agents: Problem Definition and Comparison of Baseline Algorithms
Marc Lanctot, Kate Larson, Ian Gemp +1
As intelligent agents become more generally-capable, i.e. able to master a wide variety of tasks, the complexity and cost of properly evaluating them rises significantly. Tasks tha…