973 citations · 1.2k across the 17 of their papers we have counts for
5 papers · 1 filter
EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities
Talor Abramovich, Meet Udeshi, Minghao Shao +13
Although language model (LM) agents have demonstrated increased performance in multiple domains, including coding and web-browsing, their success in cybersecurity has been limited.…
ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software
Xiang Mei, Jordi Del Castillo, Pulkit Singh Singaria +8
Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at t…
NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security
Minghao Shao, Sofija Jancheska, Meet Udeshi +10
Large Language Models (LLMs) are being deployed across various domains today. However, their capacity to solve Capture the Flag (CTF) challenges in cybersecurity has not been thoro…
Model Cascading for Code: A Cascaded Black-Box Multi-Model Framework for Cost-Efficient Code Completion with Self-Testing
Boyuan Chen, Mingzhi Zhu, Brendan Dolan-Gavitt +2
The rapid advancement of large language models (LLMs) has significantly improved code completion tasks, yet the trade-off between accuracy and computational cost remains a critical…
An Empirical Evaluation of LLMs for Solving Offensive Security Challenges
Minghao Shao, Boyuan Chen, Sofija Jancheska +4
Capture The Flag (CTF) challenges are puzzles related to computer security scenarios. With the advent of large language models (LLMs), more and more CTF participants are using LLMs…