Towards Natural Robustness Against Adversarial Examples

Abstract

Recent studies have shown that deep neural networks are vulnerable to adversarial examples, but most of the methods proposed to defense adversarial examples cannot solve this problem fundamentally. In this paper, we theoretically prove that there is an upper bound for neural networks with identity mappings to constrain the error caused by adversarial noises. However, in actual computations, this kind of neural network no longer holds any upper bound and is therefore susceptible to adversarial examples. Following similar procedures, we explain why adversarial examples can fool other deep neural networks with skip connections. Furthermore, we demonstrate that a new family of deep neural networks called Neural ODEs (Chen et al., 2018) holds a weaker upper bound. This weaker upper bound prevents the amount of change in the result from being too large. Thus, Neural ODEs have natural robustness against adversarial examples. We evaluate the performance of Neural ODEs compared with ResNet under three white-box adversarial attacks (FGSM, PGD, DI2-FGSM) and one black-box adversarial attack (Boundary Attack). Finally, we show that the natural robustness of Neural ODEs is even better than the robustness of neural networks that are trained with adversarial training methods, such as TRADES and YOPO.

References in corpus (8)

• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Theoretically Principled Trade-off between Robustness and Accuracy
• FractalNet: Ultra-Deep Neural Networks without Residuals
• A study of the effect of JPG compression on adversarial images
• Biologically inspired protection of deep networks from adversarial attacks
• Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
• Improved robustness to adversarial examples using Lipschitz regularization of the loss