18 citations · 18 across the 4 of their papers we have counts for
4 papers · 1 filter
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +306
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
Sai Kolasani, Maxim Saplin, Nicholas Crispino +5
We introduce LLM CHESS, an evaluation framework designed to probe the generalization of reasoning and instruction-following abilities in large language models (LLMs) through extend…
Budget-aware Test-time Scaling via Discriminative Verification
Kyle Montgomery, Sijun Tan, Yuqi Chen +4
Test-time scaling is a powerful strategy for boosting the performance of large language models on complex reasoning tasks. While state-of-the-art approaches often employ generative…
JudgeBench: A Benchmark for Evaluating LLM-based Judges
Sijun Tan, Siyuan Zhuang, Kyle Montgomery +5
LLM-based judges have emerged as a scalable alternative to human evaluation and are increasingly used to assess, compare, and improve models. However, the reliability of LLM-based…