6 papers
Disentangling Task Difficulty from Run-Level Failure in Agent Failure Prediction
Mohsen EsfandyariDoulabi, Lawrence Arkoh, Biruk Tadesse +4
Predicting whether an LLM agent will fail has emerged as a promising direction for supporting intervention during execution. Recent approaches report strong predictive performance,…
Evaluating Language Models on Cross-Language Code Functional Equivalence
Hui Sun, Anderson Uchôa, Rohit Gheyi +1
Background: Large Language Models (LLMs) have demonstrated strong performance across a variety of code-understanding tasks, leading many to believe that they can reason about progr…
Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure
Tural Mehtiyev, Wesley Assunção
Coding agents represent a new paradigm in automated software engineering, combining the reasoning capabilities of Large Language Models (LLMs) with tool-augmented interaction loops…
Where are the Hidden Gems? Applying Transformer Models for Design Discussion Detection
Lawrence Arkoh, Daniel Feitosa, Wesley K. G. Assunção
Design decisions are at the core of software engineering and appear in Q\&A forums, mailing lists, pull requests, issue trackers, and commit messages. Design discussions spanning a…
Test Code Review in the Era of GitHub Actions: A Replication Study
Hui Sun, Yinan Wu, Wesley K. G. Assunção +1
Test code is indispensable in software development, ensuring the correctness of production code and supporting maintainability. Nonetheless, errors or omissions in the test code ca…
RefModel: Detecting Refactorings using Foundation Models
Pedro Simões, Rohit Gheyi, Rian Melo +3
Refactoring is a common software engineering practice that improves code quality without altering program behavior. Although tools like ReExtractor+, RefactoringMiner, and RefDiff…