1 citations · 3 across the 19 of their papers we have counts for
23 papers
What Does an Agentic Software Engineering Benchmark Measure? Profiling Task Demands and Agent Behaviour Beyond What Category Labels Reveal
Radin Shayanfar, Keheliya Gallaba, Ahmed E. Hassan
Agentic software engineering benchmarks are typically summarized by nominal category labels such as "bug fix" or "feature implementation," yet benchmarks carrying the same label ar…
LLM Post-Training as Brownfield Maintenance: An Industrial Perspective on Dataware Engineering
Gopi Krishnan Rajbahadur, Amir M. Ebrahimi, Boyuan Chen +1
Industrial post-training is a brownfield regime. Teams inherit a deployed checkpoint and must land targeted improvements under fixed compute and mixture budgets without regressing…
To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia +2
Large language models increasingly write and repair production code, yet evidence is mounting that their test-passing patches leave codebases harder to maintain. We identify one co…
MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis
Yihao Chen, Shi Chang, Khaled Chawa +4
Coding agents have made substantial progress on software engineering tasks that modify existing codebases, including bug fixing and feature implementation. However, constructing a…
When Elo Lies: Hidden Biases in Codeforces-Based Evaluation of Large Language Models
Shenyu Zheng, Ximing Dong, Xiaoshuang Liu +6
As Large Language Models (LLMs) achieve breakthroughs in complex reasoning, Codeforces-based Elo ratings have emerged as a prominent metric for evaluating competitive programming c…
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
Ximing Dong, Shaowei Wang, Dayi Lin +2
Large Language Models (LLMs) achieve strong performance across many tasks but suffer from high inference latency due to autoregressive decoding. The issue is exacerbated in Large R…
SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
Zhiyu Fan, Kirill Vasilevski, Dayi Lin +6
The advancement of large language models (LLMs) and code agents has demonstrated significant potential to assist software engineering (SWE) tasks, such as autonomous issue resoluti…
RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale
Zhilong Chen, Chengzong Zhao, Boyuan Chen +9
Training software engineering (SWE) LLMs is bottlenecked by expensive infrastructure, inefficient evaluation pipelines, scarce training data, and costly quality control. We present…
Towards Reliable Generation of Executable Workflows by Foundation Models
Sogol Masoumzadeh, Keheliya Gallaba, Dayi Lin +1
Recent advancements in Foundation Models (FMs) have demonstrated significant progress in processing complex natural language to perform intricate tasks. Successfully executing thes…
An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications
Mohammed Mehedi Hasan, Hao Li, Emad Fallahzadeh +3
Foundation model (FM)-based AI agents are rapidly gaining adoption across diverse domains, but their inherent non-determinism and non-reproducibility pose testing and quality assur…
SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation
Gustavo A. Oliva, Gopi Krishnan Rajbahadur, Aaditya Bhatia +7
High-quality labeled datasets are crucial for training and evaluating foundation models in software engineering, but creating them is often prohibitively expensive and labor-intens…
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
Kirill Vasilevski, Benjamin Rombaut, Gopi Krishnan Rajbahadur +10
Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components. In this KD…
Characterizing Multi-Hunk Patches: Divergence, Proximity, and LLM Repair Challenges
Noor Nashid, Daniel Ding, Keheliya Gallaba +2
Multi-hunk bugs, where fixes span disjoint regions of code, are common in practice, yet remain underrepresented in automated repair. Existing techniques and benchmarks pre-dominant…
Towards Conversational Development Environments: Using Theory-of-Mind and Multi-Agent Architectures for Requirements Refinement
Keheliya Gallaba, Ali Arabat, Dayi Lin +2
Foundation Models (FMs) have shown remarkable capabilities in various natural language tasks. However, their ability to accurately capture stakeholder requirements remains a signif…
Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization
Ximing Dong, Shaowei Wang, Dayi Lin +1
Optimizing Large Language Model (LLM) performance requires well-crafted prompts, but manual prompt engineering is labor-intensive and often ineffective. Automated prompt optimizati…
LicenseGPT: A Fine-tuned Foundation Model for Publicly Available Dataset License Compliance
Jingwen Tan, Gopi Krishnan Rajbahadur, Zi Li +5
Dataset license compliance is a critical yet complex aspect of developing commercial AI products, particularly with the increasing use of publicly available datasets. Ambiguities i…
Engineering AI Judge Systems
Jiahuei Lin, Dayi Lin, Sky Zhang +1
AI judge systems are designed to automatically evaluate Foundation Model-powered software (i.e., FMware). Due to the intrinsic dynamic and stochastic nature of FMware, the developm…
Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models
Kirill Vasilevski, Dayi Lin, Ahmed E. Hassan
To balance the quality and inference cost of a Foundation Model (FM, such as large language models (LLMs)) powered software, people often opt to train a routing model that routes r…
PromptExp: Multi-granularity Prompt Explanation of Large Language Models
Ximing Dong, Shaowei Wang, Dayi Lin +4
Large Language Models excel in tasks like natural language understanding and text generation. Prompt engineering plays a critical role in leveraging LLM effectively. However, LLMs…
From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap
Gopi Krishnan Rajbahadur, Gustavo A. Oliva, Dayi Lin +2
The rapid expansion of foundation models (FMs), such as large language models (LLMs), has given rise to FMware, software systems that integrate FM(s) as core components. While buil…
Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap
Ahmed E. Hassan, Gustavo A. Oliva, Dayi Lin +3
The rise of AI-assisted software engineering (SE 2.0), powered by Foundation Models (FMs) and FM-powered coding assistants, has shown promise in improving developer productivity. H…
Data Quality Antipatterns for Software Analytics
Aaditya Bhatia, Dayi Lin, Gopi Krishnan Rajbahadur +2
Background: Data quality is vital in software analytics, particularly for machine learning (ML) applications like software defect prediction (SDP). Despite the widespread use of ML…
A Framework for Real-time Safeguarding the Text Generation of Large Language Model
Ximing Dong, Dayi Lin, Shaowei Wang +1
Large Language Models (LLMs) have significantly advanced natural language processing (NLP) tasks but also pose ethical and societal risks due to their propensity to generate harmfu…