Fooling Vision and Language Models Despite Localization and Attention Mechanism

Abstract

Adversarial attacks are known to succeed on classifiers, but it has been an open question whether more complex vision systems are vulnerable. In this paper, we study adversarial examples for vision and language models, which incorporate natural language understanding and complex structures such as attention, localization, and modular architectures. In particular, we investigate attacks on a dense captioning model and on two visual question answering (VQA) models. Our evaluation shows that we can generate adversarial examples with a high success rate (i.e., > 90%) for these models. Our work sheds new light on understanding adversarial attacks on vision systems which have a language component and shows that attention, bounding box localization, and compositional internal structures are vulnerable to adversarial attacks. These observations will inform future work towards building effective defenses.

arXiv commentCVPR 2018

References in corpus (26)

• Adam: A Method for Stochastic Optimization
• Ensemble Adversarial Training: Attacks and Defenses
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• VQA: Visual Question Answering
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Dynamic Memory Networks for Visual and Textual Question Answering
• Synthesizing Robust Adversarial Examples
• The Space of Transferable Adversarial Examples
• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
• Adversarial Machine Learning at Scale
• On the (Statistical) Detection of Adversarial Examples
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
• Defensive Distillation is Not Robust to Adversarial Examples
• MagNet: a Two-Pronged Defense against Adversarial Examples
• Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
• Delving into adversarial attacks on deep policies
• Learning to Compose Neural Networks for Question Answering
• Learning to Reason: End-to-End Module Networks for Visual Question Answering
• Practical Black-Box Attacks against Machine Learning
• Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
• Adversarial Examples for Semantic Segmentation and Object Detection
• Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
• Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
• Adversarial Examples for Semantic Image Segmentation
• Standard detectors aren't (currently) fooled by physical adversarial stop signs

Cited by in corpus (5)

• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions
• Adversarial Learning of Deepfakes in Accounting
• Adversarial Attacks Beyond the Image Space
• Query-Efficient Black-Box Attack by Active Learning