3 citations · 6 across the 14 of their papers we have counts for
10 papers · 1 filter
How Do LLM Agents Actually Get the Flag? Trace-Level Provenance for Agentic Offensive Security Evaluation
Kimberly Milner, Minghao Shao, Nanda Rani +8
Capture-the-Flag (CTF) benchmarks are widely used to assess the offensive security capabilities of autonomous language-model agents. Evaluations rely on shallow binary judgments or…
Safeguarding LLMs Against Misuse and AI-Driven Malware Using Steganographic Canaries
Md Raz, Venkata Sai Charan Putrevu, Meet Udeshi +3
AI-powered malware increasingly exploits cloud-hosted generative-AI services and large language models (LLMs) as analysis engines for reconnaissance and code generation. Simultaneo…
CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking
Nanda Rani, Kimberly Milner, Minghao Shao +9
Existing benchmarks for LLM-based offensive security agents use isolated, single-target setups with a known vulnerable service and fixed objective. They measure exploitation effect…
Binary Diff Summarization using Large Language Models
Meet Udeshi, Venkata Sai Charan Putrevu, Prashanth Krishnamurthy +4
Security of software supply chains is necessary to ensure that software updates do not contain maliciously injected code or introduce vulnerabilities that may compromise the integr…
Ransomware 3.0: Self-Composing and LLM-Orchestrated
Md Raz, Meet Udeshi, P. V. Sai Charan +3
Using automated reasoning, code synthesis, and contextual decision-making, we introduce a new threat that exploits large language models (LLMs) to autonomously plan, adapt, and exe…
An Orchestrated Sandbox for Side-Channel Analysis of Cyber-Physical Systems Malware
Meet Udeshi, Venkata Sai Charan Putrevu, Prashanth Krishnamurthy +2
Embedded devices powering cyber-physical systems (CPS) are increasingly internet-exposed, placing critical infrastructure under active threat. Dynamic analysis of embedded malware…
Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark
Minghao Shao, Nanda Rani, Kimberly Milner +9
Recent advances in LLM agentic systems have improved the automation of offensive security tasks, particularly for Capture the Flag (CTF) challenges. We systematically investigate t…
CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution
Minghao Shao, Haoran Xi, Nanda Rani +9
Large Language Model (LLM) agents can automate cybersecurity tasks and can adapt to the evolving cybersecurity landscape without re-engineering. While LLM agents have demonstrated…
NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security
Minghao Shao, Sofija Jancheska, Meet Udeshi +10
Large Language Models (LLMs) are being deployed across various domains today. However, their capacity to solve Capture the Flag (CTF) challenges in cybersecurity has not been thoro…
REMaQE: Reverse Engineering Math Equations from Executables
Meet Udeshi, Prashanth Krishnamurthy, Hammond Pearce +2
Cybersecurity attacks on embedded devices for industrial control systems and cyber-physical systems may cause catastrophic physical damage as well as economic loss. This could be a…