3 papers
cs.AI2026
TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks
Zhaoyang Chu, Jiarui Hu, Xingyu Jiang +8
We introduce TerminalWorld, a scalable data engine that automatically reverse-engineers high-fidelity evaluation tasks from "in-the-wild" terminal recordings. Processing 80,870 ter…
cs.LG2026
ContextBench: A Benchmark for Context Retrieval in Coding Agents
Han Li, Letian Zhu, Bohan Zhang +7
LLM-based coding agents have shown strong performance on automated issue resolution benchmarks, yet existing evaluations largely focus on final task success, providing limited insi…
cs.SE2026
Prometheus: Towards Long-Horizon Codebase Navigation for Repository-Level Problem Solving
Yue Pan, Zimin Chen, Siyu Lu +8
Large Language Models (LLMs) have shown remarkable capabilities in automating software engineering tasks, spurring the emergence of coding agents that scaffold LLMs with external t…