4 papers
Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
Yuangang Li, Justin Tian Jin Chen, Ethan Yu +2
Large language models (LLMs) increasingly rely on explicit reasoning to solve coding tasks, yet evaluating the quality of this reasoning remains challenging. Existing reasoning eva…
From Bias To Improved Prompts: A Case Study of Bias Mitigation of Clone Detection Models
QiHong Chen, Lianghao Jiang, Iftekhar Ahmed
The issue of clone code has persisted in software engineering, primarily because developers often copy and paste code segments. This common practice has elevated the importance of…
Human or LLM? A Comparative Study on Accessible Code Generation Capability
Hyunjae Suh, Mahan Tafreshipour, Sam Malek +1
Web accessibility is essential for inclusive digital experiences, yet the accessibility of LLM-generated code remains underexplored. This paper presents an empirical study comparin…
Does the Order of Fine-tuning Matter and Why?
Qihong Chen, Jiawei Li, Hyunjae Suh +5
To improve the performance on a target task, researchers have fine-tuned language models with an intermediate task before the target task of interest. However, previous works have…