6 papers
BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
Terry Yue Zhuo, Xiaolong Jin, Hange Liu +37
Crowdsourced model evaluation platforms, such as Chatbot Arena, enable real-time evaluation from human perspectives to assess the quality of model responses. In the coding domain,…
Identifying and Mitigating API Misuse in Large Language Models
Terry Yue Zhuo, Junda He, Jiamou Sun +4
API misuse in code generated by large language models (LLMs) presents a serious and growing challenge in software development, as although LLMs demonstrate impressive code generati…
REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers
Xingjian Leng, Jaskirat Singh, Yunzhong Hou +3
In this paper we tackle a fundamental question: "Can we train latent diffusion models together with the variational auto-encoder (VAE) tokenizer in an end-to-end manner?" Tradition…
HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities
Xiaoxue Ren, Penghao Jiang, Kaixin Li +6
Web applications are prime targets for cyberattacks as gateways to critical services and sensitive data. Traditional penetration testing is costly and expertise-intensive, making i…
Software Security Mapping Framework: Operationalization of Security Requirements
Sung Une Lee, Liming Dong, Zhenchang Xing +2
The escalating complexity of modern software development environments has heightened concerns around supply chain security. However, existing frameworks often fall short in transla…
From Code to Courtroom: LLMs as the New Software Judges
Junda He, Jieke Shi, Terry Yue Zhuo +5
Recently, Large Language Models (LLMs) have been increasingly used to automate SE tasks such as code generation and summarization. However, evaluating the quality of LLM-generated…