3 citations · 3 across the 3 of their papers we have counts for
4 papers
LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs
Lukas Twist, Twm Stone, Helen Yannakoudakis +1
Large language models (LLMs) have been shown to exhibit strong Python preferences when generating project-level code, but there is currently no systematic way to measure this behav…
A Study of LLMs' Preferences for Libraries and Programming Languages
Lukas Twist, Mark Harman, Don Syme +4
Despite the rapid progress of large language models (LLMs) in code generation, existing evaluations focus on functional correctness or syntactic validity, overlooking how LLMs make…
Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries
Lukas Twist, Jie M. Zhang, Mark Harman +1
Large language models (LLMs) now play a central role in code generation, yet they continue to hallucinate, frequently inventing non-existent libraries. Such library hallucinations…
Measuring the Influence of Incorrect Code on Test Generation
Dong Huang, Jie M. Zhang, Mark Harman +2
It is natural to suppose that a Large Language Model is more likely to generate correct test cases when prompted with correct code under test, compared to incorrect code under test…