GIM: Gaussian Isolation Machines

Abstract

In many cases, neural network classifiers are likely to be exposed to input data that is outside of their training distribution data. Samples from outside the distribution may be classified as an existing class with high probability by softmax-based classifiers; such incorrect classifications affect the performance of the classifiers and the applications/systems that depend on them. Previous research aimed at distinguishing training distribution data from out-of-distribution data (OOD) has proposed detectors that are external to the classification method. We present Gaussian isolation machine (GIM), a novel hybrid (generative-discriminative) classifier aimed at solving the problem arising when OOD data is encountered. The GIM is based on a neural network and utilizes a new loss function that imposes a distribution on each of the trained classes in the neural network's output space, which can be approximated by a Gaussian. The proposed GIM's novelty lies in its discriminative performance and generative capabilities, a combination of characteristics not usually seen in a single classifier. The GIM achieves state-of-the-art classification results on image recognition and sentiment analysis benchmarking datasets and can also deal with OOD inputs.

arXiv commentSubmitted to IJCNN2020 conference

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Improving neural networks by preventing co-adaptation of feature detectors
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Tutorial on Variational Autoencoders
• Activation Functions: Comparison of trends in Practice and Research for Deep Learning
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• Robust Large Margin Deep Neural Networks
• Anomaly Detection using One-Class Neural Networks