3 papers
cs.SE2026
MigrationBench: Repository-Level Code Migration Benchmark from Java 8
Linbo Liu, Xinle Liu, Qiang Zhou +8
With the rapid advancement of powerful large language models (LLMs) in recent years, a wide range of software engineering tasks can now be addressed using LLMs, significantly enhan…
cs.AI2025
Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Models
Simeng Han, Howard Dai, Stephen Xia +7
Accuracy remains a standard metric for evaluating AI systems, but it offers limited insight into how models arrive at their solutions. In this work, we introduce a benchmark based…
cs.CL2025
Large Language Model Critics for Execution-Free Evaluation of Code Changes
Aashish Yadavally, Hoan Nguyen, Laurent Callot +1
Large language models (LLMs) offer a promising way forward for automating software engineering tasks, such as bug fixes, feature additions, etc., via multi-step LLM-based agentic w…