1 citations · 1 across the 9 of their papers we have counts for
11 papers
Decentralized LLM Workflows via Sequential Auctions
Ian Gemp, Shubhendu Trivedi, Adam Fisch +7
We present a scalable framework for solving multi-step tasks through a decentralized market of self-interested agents. Prior theoretical work demonstrated that agents organized thr…
Game Arena: Strategic LLM Evaluation in Competitive Environments
Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena…
Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
Adam Fisch, Shubhendu Trivedi, Fantine Huot +5
Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist w…
Strategy, Not Payoffs: A Behavioural Embedding of Normal-Form Games
Joshua Caiata, Sreepriya Pulyassary, Xiang Li +1
Learning a strategic task changes more than what is directly taught: fine-tuning on one game can either enhance or degrade an agent's ability to reason in another. Understanding an…
Active Evaluation of General Agents: Problem Definition and Comparison of Baseline Algorithms
Marc Lanctot, Kate Larson, Ian Gemp +1
As intelligent agents become more generally-capable, i.e. able to master a wide variety of tasks, the complexity and cost of properly evaluating them rises significantly. Tasks tha…
The Alignment Game: A Theory of Long-Horizon Alignment Through Recursive Curation
Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson +1
In self-consuming generative models that train on their own outputs, alignment with user preferences becomes a recursive rather than one-time process. We provide the first formal f…
Your Recourse, My Loss? Algorithmic Recourse under Shared Constraints
Zahra Khotanlou, Kate Larson, Amir-Hossein Karimi
Decision makers are increasingly relying on machine learning in sensitive situations. Algorithmic recourse aims to provide individuals with actionable and minimally costly steps to…
Jackpot! Alignment as a Maximal Lottery
Roberto-Rafael Maura-Rivero, Marc Lanctot, Francesco Visin +1
Reinforcement Learning from Human Feedback (RLHF), the standard for aligning Large Language Models (LLMs) with human values, is known to fail to satisfy properties that are intuiti…
Soft Condorcet Optimization for Ranking of General Agents
Marc Lanctot, Kate Larson, Michael Kaisers +7
Driving progress of AI models and agents requires comparing their performance on standardized benchmarks; for general agents, individual performances must be aggregated across a po…
Approximating the Core via Iterative Coalition Sampling
Ian Gemp, Marc Lanctot, Luke Marris +11
The core is a central solution concept in cooperative game theory, defined as the set of feasible allocations or payments such that no subset of agents has incentive to break away…
Evaluating Agents using Social Choice Theory
Marc Lanctot, Kate Larson, Yoram Bachrach +6
We argue that many general evaluation problems can be viewed through the lens of voting theory. Each task is interpreted as a separate voter, which requires only ordinal rankings o…