Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

Shahin Honarvar, Amber Gorzynski, James Lee-Jones +4

Agentic large language models (LLMs) are increasingly evaluated on cybersecurity tasks using capture-the-flag (CTF) benchmarks, yet existing pointwise benchmarks offer limited insi…