Denoising Autoencoders for Overgeneralization in Neural Networks

Abstract

Despite the recent developments that allowed neural networks to achieve impressive performance on a variety of applications, these models are intrinsically affected by the problem of overgeneralization, due to their partitioning of the full input space into the fixed set of target classes used during training. Thus it is possible for novel inputs belonging to categories unknown during training or even completely unrecognizable to humans to fool the system into classifying them as one of the known classes, even with a high degree of confidence. Solving this problem may help improve the security of such systems in critical applications, and may further lead to applications in the context of open set recognition and 1-class recognition. This paper presents a novel way to compute a confidence score using denoising autoencoders and shows that such confidence score can correctly identify the regions of the input space close to the training distribution by approximately identifying its local maxima.

9 pages, 5 figures, submitted

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Energy-based Generative Adversarial Network
• Towards Open World Recognition
• Anomaly Detection with Score functions based on Nearest Neighbor Graphs

Cited by in corpus (8)

• Collective Decision of One-vs-Rest Networks for Open Set Recognition
• Perceptual Image Anomaly Detection
• One-vs-Rest Network-based Deep Probability Model for Open Set Recognition
• Analysis of Confident-Classifiers for Out-of-distribution Detection
• Controlling Over-generalization and its Effect on Adversarial Examples Generation and Detection
• Synthetic Unknown Class Learning for Learning Unknowns
• Teacher-Explorer-Student Learning: A Novel Learning Method for Open Set Recognition
• Detecting Anomalies Through Contrast in Heterogeneous Data