6 citations · 19 across the 21 of their papers we have counts for
16 papers · 1 filter
LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation
Han Li, Zhemin Fang, Rili Feng +8
Coding agent infrastructure is shifting from harness engineering toward loop engineering as coding agents are deployed for sustained long-horizon software development. Existing ben…
Beyond Correctness: Enhancing Architectural Reasoning in Code LLMs via Scalable Labeling with Agentic Judgment
Kirill Vasilevski, Ximing Dong, Benjamin Rombaut +8
LLMs have substantially improved software engineering yet real-world development requires architectural understanding. Such understanding is prohibitively expensive to label manual…
When Elo Lies: Hidden Biases in Codeforces-Based Evaluation of Large Language Models
Shenyu Zheng, Ximing Dong, Xiaoshuang Liu +6
As Large Language Models (LLMs) achieve breakthroughs in complex reasoning, Codeforces-based Elo ratings have emerged as a prominent metric for evaluating competitive programming c…
SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
Zhiyu Fan, Kirill Vasilevski, Dayi Lin +6
The advancement of large language models (LLMs) and code agents has demonstrated significant potential to assist software engineering (SWE) tasks, such as autonomous issue resoluti…
RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale
Zhilong Chen, Chengzong Zhao, Boyuan Chen +9
Training software engineering (SWE) LLMs is bottlenecked by expensive infrastructure, inefficient evaluation pipelines, scarce training data, and costly quality control. We present…
Towards Reliable Generation of Executable Workflows by Foundation Models
Sogol Masoumzadeh, Keheliya Gallaba, Dayi Lin +1
Recent advancements in Foundation Models (FMs) have demonstrated significant progress in processing complex natural language to perform intricate tasks. Successfully executing thes…
Agentic Software Engineering: Foundational Pillars and a Research Roadmap
Ahmed E. Hassan, Hao Li, Dayi Lin +4
Agentic Software Engineering (SE 3.0) represents a new era where intelligent agents are tasked not with simple code generation, but with achieving complex, goal-oriented SE objecti…
SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation
Gustavo A. Oliva, Gopi Krishnan Rajbahadur, Aaditya Bhatia +7
High-quality labeled datasets are crucial for training and evaluating foundation models in software engineering, but creating them is often prohibitively expensive and labor-intens…
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
Kirill Vasilevski, Benjamin Rombaut, Gopi Krishnan Rajbahadur +10
Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components. In this KD…
Towards Conversational Development Environments: Using Theory-of-Mind and Multi-Agent Architectures for Requirements Refinement
Keheliya Gallaba, Ali Arabat, Dayi Lin +2
Foundation Models (FMs) have shown remarkable capabilities in various natural language tasks. However, their ability to accurately capture stakeholder requirements remains a signif…
Engineering AI Judge Systems
Jiahuei Lin, Dayi Lin, Sky Zhang +1
AI judge systems are designed to automatically evaluate Foundation Model-powered software (i.e., FMware). Due to the intrinsic dynamic and stochastic nature of FMware, the developm…
From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap
Gopi Krishnan Rajbahadur, Gustavo A. Oliva, Dayi Lin +2
The rapid expansion of foundation models (FMs), such as large language models (LLMs), has given rise to FMware, software systems that integrate FM(s) as core components. While buil…
Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap
Ahmed E. Hassan, Gustavo A. Oliva, Dayi Lin +3
The rise of AI-assisted software engineering (SE 2.0), powered by Foundation Models (FMs) and FM-powered coding assistants, has shown promise in improving developer productivity. H…
Data Quality Antipatterns for Software Analytics
Aaditya Bhatia, Dayi Lin, Gopi Krishnan Rajbahadur +2
Background: Data quality is vital in software analytics, particularly for machine learning (ML) applications like software defect prediction (SDP). Despite the widespread use of ML…
Rethinking Software Engineering in the Foundation Model Era: From Task-Driven AI Copilots to Goal-Driven AI Pair Programmers
Ahmed E. Hassan, Gustavo A. Oliva, Dayi Lin +3
The advent of Foundation Models (FMs) and AI-powered copilots has transformed the landscape of software development, offering unprecedented code completion capabilities and enhanci…
Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware
Ahmed E. Hassan, Dayi Lin, Gopi Krishnan Rajbahadur +9
Foundation models (FMs), such as Large Language Models (LLMs), have revolutionized software development by enabling new use cases and business models. We refer to software built us…