CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
arXiv:2102.05311
Abstract
We investigate the adversarial robustness of CNNs from the perspective of channel-wise activations. By comparing \textit{non-robust} (normally trained) and \textit{robustified} (adversarially trained) models, we observe that adversarial training (AT) robustifies CNNs by aligning the channel-wise activations of adversarial data with those of their natural counterparts. However, the channels that are \textit{negatively-relevant} (NR) to predictions are still over-activated when processing adversarial data. Besides, we also observe that AT does not result in similar robustness for all classes. For the robust classes, channels with larger activation magnitudes are usually more \textit{positively-relevant} (PR) to predictions, but this alignment does not hold for the non-robust classes. Given these observations, we hypothesize that suppressing NR channels and aligning PR ones with their relevances further enhances the robustness of CNNs under AT. To examine this hypothesis, we introduce a novel mechanism, i.e., \underline{C}hannel-wise \underline{I}mportance-based \underline{F}eature \underline{S}election (CIFS). The CIFS manipulates channels' activations of certain layers by generating non-negative multipliers to these channels based on their relevances to predictions. Extensive experiments on benchmark datasets including CIFAR10 and SVHN clearly verify the hypothesis and CIFS's effectiveness of robustifying CNNs. \url{https://github.com/HanshuYAN/CIFS}
References in corpus (15)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- A Survey of Deep Learning Techniques for Autonomous Driving
- Theoretically Principled Trade-off between Robustness and Accuracy
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
- On the Convergence and Robustness of Adversarial Training
- Sparse DNNs with Improved Adversarial Robustness
- Attacks Which Do Not Kill Training Make Adversarial Learning Stronger
- Improving Adversarial Robustness via Channel-wise Activation Suppressing
- On Robustness of Neural Ordinary Differential Equations
- Interpreting Adversarial Examples by Activation Promotion and Suppression
- Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing
- Curriculum Adversarial Training
- Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
- Robustness via Deep Low-Rank Representations
- RAIN: A Simple Approach for Robust and Accurate Image Classification Networks