Beyond the Verdict: Evidence-Aligned Evaluation of Visual Prompt-Injection Guardrails

Suyoung Lee, Myungsub Choi

Verdict-only evaluation does not reveal whether a vision-language model (VLM) used the visual evidence that should support its decision. We study this problem in web-agent guardrai…