3 citations · 3 across the 5 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.SE2024
Commit0: Library Generation from Scratch
Wenting Zhao, Nan Jiang, Celine Lee +4
With the goal of benchmarking generative systems beyond expert software development ability, we introduce Commit0, a benchmark that challenges AI agents to write libraries from scr…
cs.AI2024
GameArena: Evaluating LLM Reasoning through Live Computer Games
Lanxiang Hu, Qiyu Li, Anze Xie +4
Evaluating the reasoning abilities of large language models (LLMs) is challenging. Existing benchmarks often depend on static datasets, which are vulnerable to data contamination a…
cs.LG2024★ 3 cited
RouterBench: A Benchmark for Multi-LLM Routing System
Qitian Jason Hu, Jacob Bieker, Xiuyu Li +5
As the range of applications for Large Language Models (LLMs) continues to grow, the demand for effective serving solutions becomes increasingly critical. Despite the versatility o…