One-vs-Rest Network-based Deep Probability Model for Open Set Recognition

Abstract

Unknown examples that are unseen during training often appear in real-world computer vision tasks, and an intelligent self-learning system should be able to differentiate between known and unknown examples. Open set recognition, which addresses this problem, has been studied for approximately a decade. However, conventional open set recognition methods based on deep neural networks (DNNs) lack a foundation for post recognition score analysis. In this paper, we propose a DNN structure in which multiple one-vs-rest sigmoid networks follow a convolutional neural network feature extractor. A one-vs-rest network, which is composed of rectified linear unit activation functions for the hidden layers and a single sigmoid target class output node, can maximize the ability to learn information from nonmatch examples. Furthermore, the network yields a sophisticated nonlinear features-to-output mapping that is explainable in the feature space. By introducing extreme value theory-based calibration techniques, the nonlinear and explainable mapping provides a well-grounded class membership probability models. Our experiments show that one-vs-rest networks can provide more informative hidden representations for unknown examples than the commonly used SoftMax layer. In addition, the proposed probability model outperformed the state-of-the art methods in open set classification scenarios.

16 pages, 11 figures

References in corpus (4)

• Striving for Simplicity: The All Convolutional Net
• The Extreme Value Machine
• Unseen Class Discovery in Open-world Classification
• Denoising Autoencoders for Overgeneralization in Neural Networks

Cited by in corpus (5)

• Generalized Out-of-Distribution Detection: A Survey
• Collective Decision of One-vs-Rest Networks for Open Set Recognition
• A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
• Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications
• Thinkback: Task-SpecificOut-of-Distribution Detection