Controlling Over-generalization and its Effect on Adversarial Examples Generation and Detection

Abstract

Convolutional Neural Networks (CNNs) significantly improve the state-of-the-art for many applications, especially in computer vision. However, CNNs still suffer from a tendency to confidently classify out-distribution samples from unknown classes into pre-defined known classes. Further, they are also vulnerable to adversarial examples. We are relating these two issues through the tendency of CNNs to over-generalize for areas of the input space not covered well by the training set. We show that a CNN augmented with an extra output class can act as a simple yet effective end-to-end model for controlling over-generalization. As an appropriate training set for the extra class, we introduce two resources that are computationally efficient to obtain: a representative natural out-distribution set and interpolated in-distribution samples. To help select a representative natural out-distribution set among available ones, we propose a simple measurement to assess an out-distribution set's fitness. We also demonstrate that training such an augmented CNN with representative out-distribution natural datasets and some interpolated samples allows it to better handle a wide range of unseen out-distribution samples and black-box adversarial examples without training it on any adversaries. Finally, we show that generation of white-box adversarial attacks using our proposed augmented CNN can become harder, as the attack algorithms have to get around the rejection regions when generating actual adversaries.

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Ensemble Adversarial Training: Attacks and Defenses
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Denoising Autoencoders for Overgeneralization in Neural Networks

Cited by in corpus (5)

• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis
• White Paper Machine Learning in Certified Systems
• Robust Training with Ensemble Consensus
• Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection