Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions

Abstract

Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial attacks. To this end, many defense approaches that attempt to improve the robustness of DNNs have been proposed. In a separate and yet related area, recent works have explored to quantize neural network weights and activation functions into low bit-width to compress model size and reduce computational complexity. In this work, we find that these two different tracks, namely the pursuit of network compactness and robustness, can be merged into one and give rise to networks of both advantages. To the best of our knowledge, this is the first work that uses quantization of activation functions to defend against adversarial examples. We also propose to train robust neural networks by using adaptive quantization techniques for the activation functions. Our proposed Dynamic Quantized Activation (DQA) is verified through a wide range of experiments with the MNIST and CIFAR-10 datasets under different white-box attack methods, including FGSM, PGD, and C & W attacks. Furthermore, Zeroth Order Optimization and substitute model-based black-box attacks are also considered in this work. The experimental results clearly show that the robustness of DNNs could be greatly improved using the proposed DQA.

References in corpus (22)

• Ensemble Adversarial Training: Attacks and Defenses
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Certified Defenses against Adversarial Examples
• Countering Adversarial Images using Input Transformations
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Stochastic Activation Pruning for Robust Adversarial Defense
• Mitigating Adversarial Effects Through Randomization
• Delving into adversarial attacks on deep policies
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
• L2-Nonexpansive Neural Networks
• Defending Against Adversarial Attacks by Leveraging an Entire GAN
• Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
• Fooling Vision and Language Models Despite Localization and Attention Mechanism
• Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models

Cited by in corpus (9)

• A Roadmap Towards Resilient Internet of Things for Cyber-Physical Systems
• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• Defensive Quantization: When Efficiency Meets Robustness
• Tensor Normalization and Full Distribution Training
• Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness
• EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks
• A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness
• Polarizing Front Ends for Robust CNNs