Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution

Wanjing Han, Levi Taiji Li, Mu Zhang +2

Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red-teaming…