Visual-Semantic Embedding Model Informed by Structured Knowledge
arXiv:2009.10026
Abstract
We propose a novel approach to improve a visual-semantic embedding model by incorporating concept representations captured from an external structured knowledge base. We investigate its performance on image classification under both standard and zero-shot settings. We propose two novel evaluation frameworks to analyse classification errors with respect to the class hierarchy indicated by the knowledge base. The approach is tested using the ILSVRC 2012 image dataset and a WordNet knowledge base. With respect to both standard and zero-shot image classification, our approach shows superior performance compared with the original approach, which uses word embeddings.
arXiv commentEuropean Starting AI Researchers' Symposium 2020 (STAIRS 2020)
References in corpus (6)
- Semi-Supervised Classification with Graph Convolutional Networks
- Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
- Zero-Shot Learning by Convex Combination of Semantic Embeddings
- Deep Learning: A Critical Appraisal
- Recent Advance in Content-based Image Retrieval: A Literature Survey
- Semi-supervised Vocabulary-informed Learning