8 citations · 8 across the 10 of their papers we have counts for
14 papers
VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
Jianing Yin, Ziyi Chen, Hao Wang +4
Agents need reliable ways to validate their outputs, both when completing tasks and when filtering synthetic training data. We study verifier generation: from the instruction and e…
Drafting Agents, Not Just Tokens: Speculative Recursive Self-Improvement for Agentic Systems
Xin Xu, Ke Cheng, Hao Wang +11
Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful sequences of edits, while full…
TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution
Shuangjie Yao, Hao Wang, Koushik Sen +3
Large language model (LLM) agents are rapidly reshaping software engineering, accompanied by an explosion of new code benchmarks. Yet nearly all existing benchmarks still rely on t…
BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
Shenghan Zheng, Zonglin Di, Yimin Liu +19
LM-agent benchmarks increasingly function as interactive evaluation infrastructure. Agents observe state, call tools, modify workspaces, submit artifacts, and receive rewards from…
ZhuLong: Execution-Grounded LLM Agent for EDA Scripting with Offline API Self-Exploration
Yang Liu, Shiwei Hou, Xiyuan Chen +13
EDA scripting with tool-specific, often undocumented APIs remains a long-tail bottleneck that existing LLMs fail to address. This paper presents ZhuLong, an execution-grounded LLM…
RECEIPT: Deterministic, Reward-Hacking-Resistant Verification for White-Box Agentic XSS Discovery
Muxi Lyu, Karen Shieh, Yiwei Hou +3
Cross-Site Scripting (XSS) remains one of the most prevalent and damaging classes of web vulnerabilities. LLM-based coding agents offer a promising approach to XSS discovery by com…
Revelio: Cost-Efficient Agentic Memory Safety Vulnerability Detection For Repository-Scale Codebases
Yiwei Hou, Hao Wang, Muxi Lyu +6
Memory safety vulnerabilities remain a significant threat even for projects with extensive fuzzing and manual auditing. Recent results suggest that large language models hold great…
Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agent
Hao Wang, Ligong Han, Kai Xu +1
Today's agents are highly effective at implementing well-scoped software design plans, but user intent is often vague and admits multiple equally valid solutions. In this paper, we…
Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
Hao Wang, Hanchen Li, Qiuyang Mang +3
Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a s…
StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
Hao Wang, Rui Li, Lei Sha +1
Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inc…
Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows
Tao Yu, Hao Wang, Changyu Li +15
Large language model (LLM) agents are increasingly expected to operate in enterprise environments, where work is distributed across specialized roles, permission-controlled systems…
SecPI: Secure Code Generation with Reasoning Models via Security Reasoning Internalization
Hao Wang, Niels Mündler, Mark Vero +3
Reasoning language models (RLMs) are increasingly used in programming. Yet, even state-of-the-art RLMs frequently introduce critical security vulnerabilities in generated code. Pri…
SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflows
Rebecca Saul, Hao Wang, Koushik Sen +1
Large language models (LLMs) have seen widespread success in code generation tasks for different scenarios, both everyday and professional. However current LLMs, despite producing…
Type-Constrained Code Generation with Language Models
Niels Mündler, Jingxuan He, Hao Wang +3
Large language models (LLMs) have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedu…