Towards an Understanding of Neural Networks in Natural-Image Spaces
arXiv:1801.09097
Abstract
Two major uncertainties, dataset bias and adversarial examples, prevail in state-of-the-art AI algorithms with deep neural networks. In this paper, we present an intuitive explanation for these issues as well as an interpretation of the performance of deep networks in a natural-image space. The explanation consists of two parts: the philosophy of neural networks and a hypothetical model of natural-image spaces. Following the explanation, we 1) demonstrate that the values of training samples differ, 2) provide incremental boost to the accuracy of a CIFAR-10 classifier by introducing an additional "random-noise" category during training, 3) alleviate over-fitting thereby enhancing the robustness against adversarial examples by detecting and excluding illusive training samples that are consistently misclassified. Our overall contribution is therefore twofold. First, while most existing algorithms treat data equally and have a strong appetite for more data, we demonstrate in contrast that an individual datum can sometimes have disproportionate and counterproductive influence and that it is not always better to train neural networks with more data. Next, we consider more thoughtful strategies by taking into account the geometric and topological properties of natural-image spaces to which deep networks are applied.
References in corpus (13)
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Focal Loss for Dense Object Detection
- NIPS 2016 Tutorial: Generative Adversarial Networks
- Robust Physical-World Attacks on Deep Learning Models
- Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
- Generative Visual Manipulation on the Natural Image Manifold
- Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
- Fast Feature Fool: A data independent approach to universal adversarial perturbations
- Universal adversarial perturbations
- Improving the Robustness of Deep Neural Networks via Stability Training
- A Testbed for Cross-Dataset Analysis
- Classification regions of deep neural networks