The More You Know: Using Knowledge Graphs for Image Classification

Abstract

One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.

CVPR 2017

References in corpus (2)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Convolutional Networks on Graph-Structured Data

Cited by in corpus (22)

• Graph Partition Neural Networks for Semi-Supervised Classification
• Iterative Visual Reasoning Beyond Convolutions
• Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey
• All About Knowledge Graphs for Actions
• Intrinsic Relationship Reasoning for Small Object Detection
• Learning Category Correlations for Multi-label Image Recognition with Graph Networks
• Bridging Knowledge Graphs to Generate Scene Graphs
• Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing
• MMKG: Multi-Modal Knowledge Graphs
• GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing
• Knowledge Graph Embedding with Multiple Relation Projections
• Context-Aware Zero-Shot Recognition
• Multi-label Zero-shot Classification by Learning to Transfer from External Knowledge
• Gaussian-Induced Convolution for Graphs
• SCR-Graph: Spatial-Causal Relationships based Graph Reasoning Network for Human Action Prediction
• HR-RCNN: Hierarchical Relational Reasoning for Object Detection
• Dynamic Graph Generation Network: Generating Relational Knowledge from Diagrams
• Few-Shot Object Detection via Knowledge Transfer
• Lift-the-flap: what, where and when for context reasoning
• Survey of Image Based Graph Neural Networks
• Meta-Path-Free Representation Learning on Heterogeneous Networks
• Wider Vision: Enriching Convolutional Neural Networks via Alignment to External Knowledge Bases