Visual Semantic Information Pursuit: A Survey

Abstract

Visual semantic information comprises two important parts: the meaning of each visual semantic unit and the coherent visual semantic relation conveyed by these visual semantic units. Essentially, the former one is a visual perception task while the latter one corresponds to visual context reasoning. Remarkable advances in visual perception have been achieved due to the success of deep learning. In contrast, visual semantic information pursuit, a visual scene semantic interpretation task combining visual perception and visual context reasoning, is still in its early stage. It is the core task of many different computer vision applications, such as object detection, visual semantic segmentation, visual relationship detection or scene graph generation. Since it helps to enhance the accuracy and the consistency of the resulting interpretation, visual context reasoning is often incorporated with visual perception in current deep end-to-end visual semantic information pursuit methods. However, a comprehensive review for this exciting area is still lacking. In this survey, we present a unified theoretical paradigm for all these methods, followed by an overview of the major developments and the future trends in each potential direction. The common benchmark datasets, the evaluation metrics and the comparisons of the corresponding methods are also introduced.

Preliminary work. Under review by IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). Do not distribute

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Loopy Belief Propagation for Approximate Inference: An Empirical Study
• Zero-Shot Learning Through Cross-Modal Transfer
• Interaction Networks for Learning about Objects, Relations and Physics
• Information Pursuit: A Bayesian Framework for Sequential Scene Parsing