Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness
arXiv:1805.10133
Abstract
For the past few years, Deep Neural Network (DNN) robustness has become a question of paramount importance. As a matter of fact, in sensitive settings misclassification can lead to dramatic consequences. Such misclassifications are likely to occur when facing adversarial attacks, hardware failures or limitations, and imperfect signal acquisition. To address this question, authors have proposed different approaches aiming at increasing the robustness of DNNs, such as adding regularizers or training using noisy examples. In this paper we propose a new regularizer built upon the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DNN architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. Since it is agnostic to the type of deformations that are expected when predicting with the DNN, the proposed regularizer can be combined with existing ad-hoc methods. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness of DNNs on classical supervised learning vision datasets.
arXiv comment12 pages, github link coming soon
References in corpus (8)
- Explaining and Harnessing Adversarial Examples
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Intriguing properties of neural networks
- mixup: Beyond Empirical Risk Minimization
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Towards Deep Neural Network Architectures Robust to Adversarial Examples
- Do CIFAR-10 Classifiers Generalize to CIFAR-10?
- PeerNets: Exploiting Peer Wisdom Against Adversarial Attacks