33 citations · 38 across the 6 of their papers we have counts for
6 papers
LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs
Lukas Twist, Twm Stone, Helen Yannakoudakis +1
Large language models (LLMs) have been shown to exhibit strong Python preferences when generating project-level code, but there is currently no systematic way to measure this behav…
Code Summaries as Diagnostic Context for LLM-Based Program Repair
Lukas Twist, Jie M. Zhang
LLMs can generate useful code, but their outputs often contain small implementation-level bugs with large behavioural effects. In this paper, we ask whether natural-language code s…
Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries
Lukas Twist, Mark Harman, Helen Yannakoudakis +1
Large language models (LLMs) now play a central role in code generation, yet they continue to hallucinate, frequently inventing non-existent libraries. Such library hallucinations…
A Study of LLMs' Preferences for Libraries and Programming Languages
Lukas Twist, Mark Harman, Don Syme +4
Despite the rapid progress of large language models (LLMs) in code generation, existing evaluations focus on functional correctness or syntactic validity, overlooking how LLMs make…
Measuring the Influence of Incorrect Code on Test Generation
Dong Huang, Jie M. Zhang, Mark Harman +2
It is natural to suppose that a Large Language Model is more likely to generate correct test cases when prompted with correct code under test, compared to incorrect code under test…
Large Language Models for Software Engineering: Survey and Open Problems
Angela Fan, Beliz Gokkaya, Mark Harman +4
This paper provides a survey of the emerging area of Large Language Models (LLMs) for Software Engineering (SE). It also sets out open research challenges for the application of LL…