Deep Feature Space: A Geometrical Perspective

Abstract

One of the most prominent attributes of Neural Networks (NNs) constitutes their capability of learning to extract robust and descriptive features from high dimensional data, like images. Hence, such an ability renders their exploitation as feature extractors particularly frequent in an abundant of modern reasoning systems. Their application scope mainly includes complex cascade tasks, like multi-modal recognition and deep Reinforcement Learning (RL). However, NNs induce implicit biases that are difficult to avoid or to deal with and are not met in traditional image descriptors. Moreover, the lack of knowledge for describing the intra-layer properties -- and thus their general behavior -- restricts the further applicability of the extracted features. With the paper at hand, a novel way of visualizing and understanding the vector space before the NNs' output layer is presented, aiming to enlighten the deep feature vectors' properties under classification tasks. Main attention is paid to the nature of overfitting in the feature space and its adverse effect on further exploitation. We present the findings that can be derived from our model's formulation, and we evaluate them on realistic recognition scenarios, proving its prominence by improving the obtained results.

arXiv commentAccepted for publication in IEEE Transactions on Pattern Analysis and Machine Intelligence

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Deep Learning Face Representation by Joint Identification-Verification
• Recent Trends in Deep Learning Based Natural Language Processing
• Additive Margin Softmax for Face Verification
• Deep Learning for Audio Signal Processing
• End-to-End Multimodal Emotion Recognition using Deep Neural Networks
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• L2-constrained Softmax Loss for Discriminative Face Verification
• Regularizing CNNs with Locally Constrained Decorrelations
• An Experimental Evaluation of Covariates Effects on Unconstrained Face Verification
• Angular Visual Hardness

Cited by in corpus (1)

• The Revisiting Problem in Simultaneous Localization and Mapping: A Survey on Visual Loop Closure Detection