5 papers
DevOps-Gym: Benchmarking AI Agents in Software DevOps Cycle
Yuheng Tang, Kaijie Zhu, Bonan Ruan +14
Even though demonstrating extraordinary capabilities in code generation and software issue resolving, AI agents' capabilities in the full software DevOps cycle are still unknown. D…
VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detection
Yuzhou Nie, Hongwei Li, Chengquan Guo +5
We propose VulnLLM-R, the~\emph{first specialized reasoning LLM} for vulnerability detection. Our key insight is that LLMs can reason about program states and analyze the potential…
SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI
Yuzhou Nie, Zhun Wang, Yu Yang +7
Existing benchmarks for evaluating the security risks and capabilities (e.g., vulnerability detection) of code-generating large language models (LLMs) face several key limitations:…
LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage
Yuzhou Nie, Zhun Wang, Ye Yu +4
Recent studies have discovered that large language models (LLM) may be ``fooled'' to output private information, including training data, system prompts, and personally identifiabl…
Data Free Backdoor Attacks
Bochuan Cao, Jinyuan Jia, Chuxuan Hu +5
Backdoor attacks aim to inject a backdoor into a classifier such that it predicts any input with an attacker-chosen backdoor trigger as an attacker-chosen target class. Existing ba…