Temporal Shuffling for Defending Deep Action Recognition Models against Adversarial Attacks

Abstract

Recently, video-based action recognition methods using convolutional neural networks (CNNs) achieve remarkable recognition performance. However, there is still lack of understanding about the generalization mechanism of action recognition models. In this paper, we suggest that action recognition models rely on the motion information less than expected, and thus they are robust to randomization of frame orders. Furthermore, we find that motion monotonicity remaining after randomization also contributes to such robustness. Based on this observation, we develop a novel defense method using temporal shuffling of input videos against adversarial attacks for action recognition models. Another observation enabling our defense method is that adversarial perturbations on videos are sensitive to temporal destruction. To the best of our knowledge, this is the first attempt to design a defense method without additional training for 3D CNN-based video action recognition models.

arXiv comment12 pages, accepted to Neural Networks

References in corpus (11)

• Explaining and Harnessing Adversarial Examples
• The Kinetics Human Action Video Dataset
• Is Space-Time Attention All You Need for Video Understanding?
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Certified Adversarial Robustness via Randomized Smoothing
• Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition
• Mitigating Adversarial Effects Through Randomization
• Adversarial Perturbations Against Real-Time Video Classification Systems
• Noise or Signal: The Role of Image Backgrounds in Object Recognition
• Denoised Smoothing: A Provable Defense for Pretrained Classifiers