Global Semantic Description of Objects based on Prototype Theory

Abstract

In this paper, we introduce a novel semantic description approach inspired on Prototype Theory foundations. We propose a Computational Prototype Model (CPM) that encodes and stores the central semantic meaning of objects category: the semantic prototype. Also, we introduce a Prototype-based Description Model that encodes the semantic meaning of an object while describing its features using our CPM model. Our description method uses semantic prototypes computed by CNN-classifications models to create discriminative signatures that describe an object highlighting its most distinctive features within the category. Our experiments show that: i) our CPM model (semantic prototype + distance metric) is able to describe the internal semantic structure of objects categories; ii) our semantic distance metric can be understood as the object visual typicality score within a category; iii) our descriptor encoding is semantically interpretable and significantly outperforms other image global encodings in clustering and classification tasks.

arXiv commentContent: 24 pages (22 + 2 reference) with 15 Figures and 3 Tables. In the future, a new version will be updated with other experiments and results (and a journal reference if applicable)

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Prototypical Networks for Few-shot Learning
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
• Do Convnets Learn Correspondence?
• Infinite Mixture Prototypes for Few-Shot Learning
• Gaussian Prototypical Networks for Few-Shot Learning on Omniglot
• Multivariate Chebyshev Inequality with Estimated Mean and Variance
• TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion
• Leveraging Class Hierarchies with Metric-Guided Prototype Learning
• End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds