26 citations · 56 across the 13 of their papers we have counts for
3 papers · 1 filter
Cost-Aware Best-LLM Identification using Dueling Feedback
Sarvesh Gharat, Nikhil Karamchandani, Jayakrishnan Nair
Inspired by the problem of identifying the best model from a collection of large language models (LLMs) with heterogeneous querying costs, we formulate and analyse a variant of the…
What Makes a Terminal-Bench Task Hard? Separating Genuine Hardness from Fake-Hardness on an Adjudicated Agentic Corpus
Edward Lue Chee Lip, Boden Moraski, Tim Knappe +4
Frontier benchmarks need tasks that current models cannot solve. But a task that no model solves is not automatically a hard task. The same zero pass rate can come from a real capa…
Representative Arm Identification: A fixed confidence approach to identify cluster representatives
Sarvesh Gharat, Aniket Yadav, Nikhil Karamchandani +1
We study the representative arm identification (RAI) problem in the multi-armed bandits (MAB) framework, wherein we have a collection of arms, each associated with an unknown rewar…