activity
20242026
most citedNYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security

3 citations · 4 across the 7 of their papers we have counts for

collaborators
Showing cs.CRShow all

5 papers · 1 filter

cs.CR2026

ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software

Xiang Mei, Jordi Del Castillo, Pulkit Singh Singaria +8

Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at t…

cs.CR2025

Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark

Minghao Shao, Nanda Rani, Kimberly Milner +9

Recent advances in LLM agentic systems have improved the automation of offensive security tasks, particularly for Capture the Flag (CTF) challenges. We systematically investigate t…

cs.CR2025

CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution

Minghao Shao, Haoran Xi, Nanda Rani +9

Large Language Model (LLM) agents can automate cybersecurity tasks and can adapt to the evolving cybersecurity landscape without re-engineering. While LLM agents have demonstrated…

cs.CR2024

ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software

Xiang Mei, Jordi Del Castillo, Pulkit Singh Singaria +8

Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at t…

cs.CR2024★ 3 cited

NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security

Minghao Shao, Sofija Jancheska, Meet Udeshi +10

Large Language Models (LLMs) are being deployed across various domains today. However, their capacity to solve Capture the Flag (CTF) challenges in cybersecurity has not been thoro…