Adversarial Attacks on Video Object Segmentation with Hard Region Discovery

Abstract

Video object segmentation has been applied to various computer vision tasks, such as video editing, autonomous driving, and human-robot interaction. However, the methods based on deep neural networks are vulnerable to adversarial examples, which are the inputs attacked by almost human-imperceptible perturbations, and the adversary (i.e., attacker) will fool the segmentation model to make incorrect pixel-level predictions. This will rise the security issues in highly-demanding tasks because small perturbations to the input video will result in potential attack risks. Though adversarial examples have been extensively used for classification, it is rarely studied in video object segmentation. Existing related methods in computer vision either require prior knowledge of categories or cannot be directly applied due to the special design for certain tasks, failing to consider the pixel-wise region attack. Hence, this work develops an object-agnostic adversary that has adversarial impacts on VOS by first-frame attacking via hard region discovery. Particularly, the gradients from the segmentation model are exploited to discover the easily confused region, in which it is difficult to identify the pixel-wise objects from the background in a frame. This provides a hardness map that helps to generate perturbations with a stronger adversarial power for attacking the first frame. Empirical studies on three benchmarks indicate that our attacker significantly degrades the performance of several state-of-the-art video object segmentation models.

References in corpus (11)

• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• One pixel attack for fooling deep neural networks
• YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
• Motion-Attentive Transition for Zero-Shot Video Object Segmentation
• Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking
• Attention-guided Temporally Coherent Video Object Matting
• Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation
• Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering
• Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentation
• Implicit Motion-Compensated Network for Unsupervised Video Object Segmentation
• Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation

Cited by in corpus (2)

• Spatial-Frequency Discriminability for Revealing Adversarial Perturbations
• Adversarial Attacks in Multimodal Systems: A Practitioner's Survey