Defective Convolutional Networks

Abstract

Robustness of convolutional neural networks (CNNs) has gained in importance on account of adversarial examples, i.e., inputs added as well-designed perturbations that are imperceptible to humans but can cause the model to predict incorrectly. Recent research suggests that the noises in adversarial examples break the textural structure, which eventually leads to wrong predictions. To mitigate the threat of such adversarial attacks, we propose defective convolutional networks that make predictions relying less on textural information but more on shape information by properly integrating defective convolutional layers into standard CNNs. The defective convolutional layers contain defective neurons whose activations are set to be a constant function. As defective neurons contain no information and are far different from standard neurons in its spatial neighborhood, the textural features cannot be accurately extracted, and so the model has to seek other features for classification, such as the shape. We show extensive evidence to justify our proposal and demonstrate that defective CNNs can defense against black-box attacks better than standard CNNs. In particular, they achieve state-of-the-art performance against transfer-based attacks without any adversarial training being applied.

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Wide Residual Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• DropBlock: A regularization method for convolutional networks
• Adversarial Examples Are Not Bugs, They Are Features
• Robustness May Be at Odds with Accuracy
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Learning with a Strong Adversary
• Comparing deep neural networks against humans: object recognition when the signal gets weaker
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation
• Metric Learning for Adversarial Robustness
• Boosting Adversarial Attacks with Momentum
• A Direct Approach to Robust Deep Learning Using Adversarial Networks
• Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser
• Adversarial Defense by Stratified Convolutional Sparse Coding