3 citations · 6 across the 7 of their papers we have counts for
8 papers
MobileCybench: Evaluating Agent Vulnerability Discovery via Executable Probes
Andy K. Zhang, Ava Huang, Joey Ji +21
AI agents now report vulnerabilities faster than maintainers can review them. Reports often depend on security properties specific to the application, and require considerable huma…
CTF for education
Yi Lyu, Luke Dotson, Nic Draves +1
In this paper, we take a close look at how CTF can be used in cybersecurity education. We divide the CTF competitions into four different categories, which are attack-based CTFs, d…
SeedAIchemy: LLM-Driven Seed Corpus Generation for Fuzzing
Aidan Wen, Norah A. Alzahrani, Jingzhi Jiang +5
We introduce SeedAIchemy, an automated LLM-driven corpus generation tool that makes it easier for developers to implement fuzzing effectively. SeedAIchemy consists of five modules…
Establishing Best Practices for Building Rigorous Agentic Benchmarks
Yuxuan Zhu, Tengjun Jin, Yada Pruksachatkun +22
Benchmarks are essential for quantitatively tracking progress in AI. As AI agents become increasingly capable, researchers and practitioners have introduced agentic benchmarks to e…
BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems
Andy K. Zhang, Joey Ji, Celeste Menders +31
AI agents have the potential to significantly alter the cybersecurity landscape. Here, we introduce the first framework to capture offensive and defensive cyber-capabilities in evo…
Frontier AI's Impact on the Cybersecurity Landscape
Yujin Potter, Wenbo Guo, Zhun Wang +6
The impact of frontier AI (i.e., AI agents and foundation models) in cybersecurity is rapidly increasing. In this paper, we comprehensively analyze this trend through multiple aspe…
Language model developers should report train-test overlap
Andy K Zhang, Kevin Klyman, Yifan Mai +4
Language models are extensively evaluated, but correctly interpreting evaluation results requires knowledge of train-test overlap which refers to the extent to which the language m…
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
Andy K. Zhang, Neil Perry, Riya Dulepet +24
Language Model (LM) agents for cybersecurity that are capable of autonomously identifying vulnerabilities and executing exploits have potential to cause real-world impact. Policyma…