6 papers
HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities
Xiaoxue Ren, Penghao Jiang, Kaixin Li +6
Web applications are prime targets for cyberattacks as gateways to critical services and sensitive data. Traditional penetration testing is costly and expertise-intensive, making i…
BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
Terry Yue Zhuo, Xiaolong Jin, Hange Liu +37
Crowdsourced model evaluation platforms, such as Chatbot Arena, enable real-time evaluation from human perspectives to assess the quality of model responses. In the coding domain,…
Software Security Mapping Framework: Operationalization of Security Requirements
Sung Une Lee, Liming Dong, Zhenchang Xing +2
The escalating complexity of modern software development environments has heightened concerns around supply chain security. However, existing frameworks often fall short in transla…
REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers
Xingjian Leng, Jaskirat Singh, Yunzhong Hou +3
In this paper we tackle a fundamental question: "Can we train latent diffusion models together with the variational auto-encoder (VAE) tokenizer in an end-to-end manner?" Tradition…
From Code to Courtroom: LLMs as the New Software Judges
Junda He, Jieke Shi, Terry Yue Zhuo +5
Recently, Large Language Models (LLMs) have been increasingly used to automate SE tasks such as code generation and summarization. However, evaluating the quality of LLM-generated…
Identifying and Mitigating API Misuse in Large Language Models
Terry Yue Zhuo, Junda He, Jiamou Sun +4
API misuse in code generated by large language models (LLMs) presents a serious and growing challenge in software development, as although LLMs demonstrate impressive code generati…