Breaking the Silence: the Threats of Using LLMs in Software Engineering

Abstract

Large Language Models (LLMs) have gained considerable traction within the Software Engineering (SE) community, impacting various SE tasks from code completion to test generation, from program repair to code summarization. Despite their promise, researchers must still be careful as numerous intricate factors can influence the outcomes of experiments involving LLMs. This paper initiates an open discussion on potential threats to the validity of LLM-based research including issues such as closed-source models, possible data leakage between LLM training data and research evaluation, and the reproducibility of LLM-based findings. In response, this paper proposes a set of guidelines tailored for SE researchers and Language Model (LM) providers to mitigate these concerns. The implications of the guidelines are illustrated using existing good practices followed by LLM providers and a practical example for SE researchers in the context of test case generation.

Accepted at the ICSE'24 conference, NIER track

References in corpus (4)

• Practical Program Repair in the Era of Large Pre-trained Language Models
• Natural Attack for Pre-trained Models of Code
• Embedding Java Classes with code2vec: Improvements from Variable Obfuscation
• Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation

Cited by in corpus (8)

• An Empirical Study of Challenges in Machine Learning Asset Management
• Generative AI to Generate Test Data Generators
• Large Language Model Powered Symbolic Execution
• Automated Detection of Algorithm Debt in Deep Learning Frameworks: An Empirical Study
• Who's Who? LLM-assisted Software Traceability with Architecture Entity Recognition
• Detecting Semantic Clones of Unseen Functionality
• Unveiling Assumptions: Exploring the Decisions of AI Chatbots and Human Testers
• RefModel: Detecting Refactorings using Foundation Models