1 citations · 1 across the 9 of their papers we have counts for
5 papers · 1 filter
Decentralized LLM Workflows via Sequential Auctions
Ian Gemp, Shubhendu Trivedi, Adam Fisch +7
We present a scalable framework for solving multi-step tasks through a decentralized market of self-interested agents. Prior theoretical work demonstrated that agents organized thr…
Game Arena: Strategic LLM Evaluation in Competitive Environments
Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena…
Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
Adam Fisch, Shubhendu Trivedi, Fantine Huot +5
Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist w…
Strategy, Not Payoffs: A Behavioural Embedding of Normal-Form Games
Joshua Caiata, Sreepriya Pulyassary, Xiang Li +1
Learning a strategic task changes more than what is directly taught: fine-tuning on one game can either enhance or degrade an agent's ability to reason in another. Understanding an…
Active Evaluation of General Agents: Problem Definition and Comparison of Baseline Algorithms
Marc Lanctot, Kate Larson, Ian Gemp +1
As intelligent agents become more generally-capable, i.e. able to master a wide variety of tasks, the complexity and cost of properly evaluating them rises significantly. Tasks tha…