Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis
arXiv:2412.14841 · doi:10.1007/s44163-026-01009-5
Abstract
Large Language Models (LLMs) are one of the most promising developments in the field of artificial intelligence, and the software engineering community has readily noticed their potential role in the software development life-cycle. Developers routinely ask LLMs to generate code snippets, increasing productivity but also potentially introducing ownership, privacy, correctness, and security issues. Previous work highlighted how code generated by mainstream commercial LLMs is often not safe, containing vulnerabilities, bugs, and code smells. In this paper, we present a framework that leverages testing and static analysis to assess the quality, and guide the self-improvement, of code generated by general-purpose, open-source LLMs. First, we ask LLMs to generate C code to solve a number of programming tasks. Then we employ ground-truth tests to assess the (in)correctness of the generated code, and a static analysis tool to detect potential safety vulnerabilities. Next, we assess the models ability to evaluate the generated code, by asking them to detect errors and vulnerabilities. Finally, we test the models ability to fix the generated code, providing the reports produced during the static analysis and incorrectness evaluation phases as feedback. Our results show that models often produce incorrect code, and that the generated code can include safety issues. Moreover, they perform very poorly at detecting either issue. On the positive side, we observe a substantial ability to fix flawed code when provided with information about failed tests or potential vulnerabilities, indicating a promising avenue for improving the safety of LLM-based code generation tools.
References in corpus (12)
- Mistral 7B
- Gemma: Open Models Based on Gemini Research and Technology
- Mixtral of Experts
- Large Language Models for Code: Security Hardening and Adversarial Testing
- The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
- Practices and Challenges of Using GitHub Copilot: An Empirical Study
- The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verification
- A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification
- PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)
- Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models
- Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning
- Can ChatGPT support software verification?