5 papers
FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics
Qiran Zou, Hou Hei Lam, Wenhao Zhao +11
AI research agents accelerate ML research by automating hypothesis generation, experimentation, and empirical refinement. Existing agent strategies range from greedy hill-climbing…
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
Bingchen Zhao, Dhruv Srikanth, Yuxiang Wu +1
As long-horizon coding agents produce more code than any developer can review, oversight collapses onto a single surface: the automated test suite. Reward hacking naturally arises…
Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap
Ahmed E. Hassan, Gustavo A. Oliva, Dayi Lin +3
The rise of AI-assisted software engineering (SE 2.0), powered by Foundation Models (FMs) and FM-powered coding assistants, has shown promise in improving developer productivity. H…
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
Kirill Vasilevski, Benjamin Rombaut, Gopi Krishnan Rajbahadur +10
Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components. In this KD…
Predicting post-release defects with knowledge units (KUs) of programming languages: an empirical study
Md Ahasanuzzaman, Gustavo A. Oliva, Ahmed E. Hassan +2
Defect prediction plays a crucial role in software engineering, enabling developers to identify defect-prone code and improve software quality. While extensive research has focused…