Graph-RISE: Graph-Regularized Image Semantic Embedding

Abstract

Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering. In this paper, we present Graph-Regularized Image Semantic Embedding (Graph-RISE), a large-scale neural graph learning framework that allows us to train embeddings to discriminate an unprecedented O(40M) ultra-fine-grained semantic labels. Graph-RISE outperforms state-of-the-art image embedding algorithms on several evaluation tasks, including image classification and triplet ranking. We provide case studies to demonstrate that, qualitatively, image retrieval based on Graph-RISE effectively captures semantics and, compared to the state-of-the-art, differentiates nuances at levels that are closer to human-perception.

9 pages, 7 figures

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning Face Representation by Joint Identification-Verification
• YouTube-8M: A Large-Scale Video Classification Benchmark

Cited by in corpus (7)

• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
• WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
• Learning Multilingual Word Embeddings Using Image-Text Data
• Multi-Image Summarization: Textual Summary from a Set of Cohesive Images
• Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection Models
• Understanding Guided Image Captioning Performance across Domains
• Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering