8 papers
LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs
Lukas Twist, Twm Stone, Helen Yannakoudakis +1
Large language models (LLMs) have been shown to exhibit strong Python preferences when generating project-level code, but there is currently no systematic way to measure this behav…
Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning
Lukas Twist, Helen Yannakoudakis, Jie M. Zhang
Explicit reasoning models are trained to produce intermediate reasoning traces before final answers, but downstream fine-tuning is often performed on ordinary instruction-response…
SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering
Jingzhi Gong, Ruizhen Gu, Zhiwei Fei +7
Agent skills are increasingly used to configure coding agents for software engineering (SE) tasks, yet current practice treats them as static, hand-crafted assets, or evolved on pa…
Not All Code Is Equal: A Data-Centric Study of Code Complexity and LLM Reasoning
Lukas Twist, Shu Yang, Hanqi Yan +4
Large Language Models (LLMs) increasingly exhibit strong reasoning abilities, often attributed to their capacity to generate chain-of-thought-style intermediate reasoning. Recent w…
A Study of Library Usage in Agent-Authored Pull Requests
Lukas Twist, Jie M. Zhang
Coding agents are becoming increasingly capable of completing end-to-end software engineering workflows that previously required a human developer, including raising pull requests…
Code Summaries as Diagnostic Context for LLM-Based Program Repair
Lukas Twist, Jie M. Zhang
LLMs can generate useful code, but their outputs often contain small implementation-level bugs with large behavioural effects. In this paper, we ask whether natural-language code s…