3 papers
cs.AI2026
ScopeBench: Do Agents Preserve Engagement Boundaries Under Goal Pressure?
Shane Caldwell, Max Harley, Ads Dawson +3
Agents are increasingly deployed with real autonomy in web application and network penetration testing, where a single out-of-scope action can breach a client's engagement boundary…
cs.CR2026
ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents
Shane Caldwell, Max Harley, Ads Dawson +3
As LLM agents take on offensive security work, a single out-of-scope tool call can breach a client's engagement boundary, disrupt production, or void a bug-bounty finding. Unlike a…
cs.AI2025
PentestJudge: Judging Agent Behavior Against Operational Requirements
Shane Caldwell, Max Harley, Michael Kouremetis +2
We introduce PentestJudge, a system for evaluating the operations of penetration testing agents. PentestJudge is a large language model (LLM)-as-judge with access to tools that all…