10 papers
ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software
Xiang Mei, Jordi Del Castillo, Pulkit Singh Singaria +8
Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at t…
ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software
Xiang Mei, Jordi Del Castillo, Pulkit Singh Singaria +8
Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at t…
CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking
Nanda Rani, Kimberly Milner, Minghao Shao +9
Existing benchmarks for LLM-based offensive security agents use isolated, single-target setups with a known vulnerable service and fixed objective. They measure exploitation effect…
Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark
Minghao Shao, Nanda Rani, Kimberly Milner +9
Recent advances in LLM agentic systems have improved the automation of offensive security tasks, particularly for Capture the Flag (CTF) challenges. We systematically investigate t…
AI In Cybersecurity Education -- Scalable Agentic CTF Design Principles and Educational Outcomes
Haoran Xi, Minghao Shao, Kimberly Milner +11
Large language models are rapidly changing how learners acquire and demonstrate cybersecurity skills. However, when human--AI collaboration is allowed, educators still lack validat…
From Trace to Line: LLM Agent for Real-World OSS Vulnerability Localization
Haoran Xi, Minghao Shao, Brendan Dolan-Gavitt +2
Large language models show promise for vulnerability discovery, yet prevailing methods inspect code in isolation, struggle with long contexts, and focus on coarse function- or file…