4 papers · 1 filter
BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
Terry Yue Zhuo, Xiaolong Jin, Hange Liu +37
Crowdsourced model evaluation platforms, such as Chatbot Arena, enable real-time evaluation from human perspectives to assess the quality of model responses. In the coding domain,…
Software Security Mapping Framework: Operationalization of Security Requirements
Sung Une Lee, Liming Dong, Zhenchang Xing +2
The escalating complexity of modern software development environments has heightened concerns around supply chain security. However, existing frameworks often fall short in transla…
From Code to Courtroom: LLMs as the New Software Judges
Junda He, Jieke Shi, Terry Yue Zhuo +5
Recently, Large Language Models (LLMs) have been increasingly used to automate SE tasks such as code generation and summarization. However, evaluating the quality of LLM-generated…
Identifying and Mitigating API Misuse in Large Language Models
Terry Yue Zhuo, Junda He, Jiamou Sun +4
API misuse in code generated by large language models (LLMs) presents a serious and growing challenge in software development, as although LLMs demonstrate impressive code generati…