3 papers
cs.SE2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini +82
AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not…
cs.AI2025
Agents of Change: Self-Evolving LLM Agents for Strategic Planning
Nikolas Belle, Dakota Barnes, Alfonso Amayuelas +3
We address the long-horizon gap in large language model (LLM) agents by enabling them to sustain coherent strategies in adversarial, stochastic environments. Settlers of Catan prov…
cs.CL2025
HardTests: Synthesizing High-Quality Test Cases for LLM Coding
Zhongmou He, Yee Man Choi, Kexun Zhang +6
Verifiers play a crucial role in large language model (LLM) reasoning, needed by post-training techniques such as reinforcement learning. However, reliable verifiers are hard to ge…