7 papers
Prompt Coverage Adequacy
Florian Tambon, Michael Konstantinou, Cedric Richter +3
In recent years, it has become increasingly evident that large language models (LLMs) and autonomous agents raise the level of abstraction in software development by shifting the f…
Reducing Hallucination in Enterprise AI Workflows via Hybrid Utility Minimum Bayes Risk (HUMBR)
Chenhao Fang, Jordi Mola, Mark Harman +10
Although LLMs drive automation, it is critical to ensure immense consideration for high-stakes enterprise workflows such as those involving legal matters, risk management, and priv…
Boosting LLMs for Mutation Generation
Bo Wang, Ming Deng, Mingda Chen +5
LLM-based mutation testing is a promising testing technology, but existing approaches typically rely on a fixed set of mutations as few-shot examples or none at all. This can resul…
Just-in-Time Catching Test Generation at Meta
Matthew Becker, Yifei Chen, Nicholas Cochran +12
We report on Just-in-Time catching test generation at Meta, designed to prevent bugs in large scale backend systems of hundreds of millions of line of code. Unlike traditional hard…
YATE: The Role of Test Repair in LLM-Based Unit Test Generation
Michael Konstantinou, Renzo Degiovanni, Jie M. Zhang +2
Recent advances in automated test generation utilises language models to produce unit tests. While effective, language models tend to generate many incorrect tests with respect to…
Harden and Catch for Just-in-Time Assured LLM-Based Software Testing: Open Research Challenges
Mark Harman, Peter O'Hearn, Shubho Sengupta
Despite decades of research and practice in automated software testing, several fundamental concepts remain ill-defined and under-explored, yet offer enormous potential real-world…