Generative AI for Test Driven Development: Preliminary Results

Abstract

Test Driven Development (TDD) is one of the major practices of Extreme Programming for which incremental testing and refactoring trigger the code development. TDD has limited adoption in the industry, as it requires more code to be developed and experienced developers. Generative AI (GenAI) may reduce the extra effort imposed by TDD. In this work, we introduce an approach to automatize TDD by embracing GenAI either in a collaborative interaction pattern in which developers create tests and supervise the AI generation during each iteration or a fully-automated pattern in which developers only supervise the AI generation at the end of the iterations. We run an exploratory experiment with ChatGPT in which the interaction patterns are compared with the non-AI TDD regarding test and code quality and development speed. Overall, we found that, for our experiment and settings, GenAI can be efficiently used in TDD, but it requires supervision of the quality of the produced code. In some cases, it can even mislead non-expert developers and propose solutions just for the sake of the query.

arXiv commentAccepted to AI4ASE workshop at XP'24

References in corpus (6)

• A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?
• AI-driven Development Is Here: Should You Worry?
• Interactive Code Generation via Test-Driven User-Intent Formalization
• Why Research on Test-Driven Development is Inconclusive?
• LLM4TDD: Best Practices for Test Driven Development Using Large Language Models
• Fixing Large Language Models' Specification Misunderstanding for Better Code Generation