4 papers
Evaluating Coding Agents on Kernel Exploit Generation
Junyoung Jang, Gwanhyun Lee, Hwiwon Lee +4
Coding agents now find real vulnerabilities in production software. However, bug discovery results do not measure whether agents can construct exploit primitives. We introduce KEX-…
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?
Hwiwon Lee, Jiawei Liu, Dongjun Kim +4
Finding a real vulnerability in complicated systems is a challenging, long-horizon task that demands reasoning across an entire codebase to produce a working proof-of-concept (PoC)…
Agentic Vulnerability Reasoning on COTS Binaries
Hwiwon Lee, Jongseong Kim, Lingming Zhang
LLM agents have been increasingly adopted for solving security tasks. However, existing evaluations usually require source code access, while commercial off-the-shelf (COTS) binari…
SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks
Hwiwon Lee, Ziqi Zhang, Hanxiao Lu +1
Rigorous security-focused evaluation of large language model (LLM) agents is imperative for establishing trust in their safe deployment throughout the software development lifecycl…