Building to the Test: Coding Agents Deliver What You Check, Not What You Requested

Yanuo Ma, Ben Kereopa-Yorke, Ben Schultz

Benchmarks are widely used to evaluate task completion by Large Language Models (LLMs), but this approach has accumulated construction-validity problems, and a passing score may no…