Deep-RBF Networks Revisited: Robust Classification with Rejection
arXiv:1812.03190
Abstract
One of the main drawbacks of deep neural networks, like many other classifiers, is their vulnerability to adversarial attacks. An important reason for their vulnerability is assigning high confidence to regions with few or even no feature points. By feature points, we mean a nonlinear transformation of the input space extracting a meaningful representation of the input data. On the other hand, deep-RBF networks assign high confidence only to the regions containing enough feature points, but they have been discounted due to the widely-held belief that they have the vanishing gradient problem. In this paper, we revisit the deep-RBF networks by first giving a general formulation for them, and then proposing a family of cost functions thereof inspired by metric learning. In the proposed deep-RBF learning algorithm, the vanishing gradient problem does not occur. We make these networks robust to adversarial attack by adding the reject option to their output layer. Through several experiments on the MNIST dataset, we demonstrate that our proposed method not only achieves significant classification accuracy but is also very resistant to various adversarial attacks.
References in corpus (3)
Cited by in corpus (6)
- Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition
- Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
- Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks
- Classification of Traffic Using Neural Networks by Rejecting: a Novel Approach in Classifying VPN Traffic
- DeepLABNet: End-to-end Learning of Deep Radial Basis Networks with Fully Learnable Basis Functions
- Improving Sample Efficiency with Normalized RBF Kernels