Zero-Shot Learning by Convex Combination of Semantic Embeddings

Abstract

Several recent publications have proposed methods for mapping images into continuous semantic embedding spaces. In some cases the embedding space is trained jointly with the image transformation. In other cases the semantic embedding space is established by an independent natural language processing task, and then the image transformation into that space is learned in a second stage. Proponents of these image embedding systems have stressed their advantages over the traditional \nway{} classification framing of image understanding, particularly in terms of the promise for zero-shot learning -- the ability to correctly annotate images of previously unseen object categories. In this paper, we propose a simple method for constructing an image embedding system from any existing \nway{} image classifier and a semantic word embedding model, which contains the $\n$ class labels in its vocabulary. Our method maps images into the semantic embedding space via convex combination of the class label embedding vectors, and requires no additional training. We show that this simple and direct method confers many of the advantages associated with more complex image embedding schemes, and indeed outperforms state of the art methods on the ImageNet zero-shot learning task.

References in corpus (1)

• Zero-Shot Learning Through Cross-Modal Transfer

Cited by in corpus (170)

• Matching Networks for One Shot Learning
• A Review of Generalized Zero-Shot Learning Methods
• Open-vocabulary Object Detection via Vision and Language Knowledge Distillation
• Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
• Learning Deep Representations of Fine-grained Visual Descriptions
• AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
• Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions
• Semantics-Guided Contrastive Network for Zero-Shot Object detection
• Local Similarity-Aware Deep Feature Embedding
• Leveraging the Invariant Side of Generative Zero-Shot Learning
• Zero-Shot Learning -- The Good, the Bad and the Ugly
• Compositional Vector Space Models for Knowledge Base Completion
• Order-Embeddings of Images and Language
• Representation Learning for Natural Language Processing
• Preserving Semantic Relations for Zero-Shot Learning
• An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
• Small Sample Learning in Big Data Era
• Background Learnable Cascade for Zero-Shot Object Detection
• Zero-Shot Learning via Class-Conditioned Deep Generative Models
• Range Loss for Deep Face Recognition with Long-tail
• Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions
• Transductive Zero-Shot Learning with Visual Structure Constraint
• Word2VisualVec: Image and Video to Sentence Matching by Visual Feature Prediction
• Ridge Regression, Hubness, and Zero-Shot Learning
• Recent Advances in Zero-shot Recognition
• Learning from Few Samples: A Survey
• Unsupervised Learning on Neural Network Outputs: with Application in Zero-shot Learning
• Zero-Shot Hashing via Transferring Supervised Knowledge
• Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs
• Deep Triplet Ranking Networks for One-Shot Recognition
• Semi-supervised Zero-Shot Learning by a Clustering-based Approach
• Discriminative Learning of Latent Features for Zero-Shot Recognition
• Semi-supervised Vocabulary-informed Learning
• Multi-modal Cycle-consistent Generalized Zero-Shot Learning
• Vocabulary-informed Zero-shot and Open-set Learning
• Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learning
• Transferable Contrastive Network for Generalized Zero-Shot Learning
• Learning the Best Pooling Strategy for Visual Semantic Embedding
• Fine-Grained Zero-Shot Learning with DNA as Side Information
• Zero-shot Learning for Audio-based Music Classification and Tagging
• Domain Adaptive Dialog Generation via Meta Learning
• Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition
• Online Lifelong Generalized Zero-Shot Learning
• Zero-Shot Learning with Generative Latent Prototype Model
• Integrating Semantic Knowledge to Tackle Zero-shot Text Classification
• Relational Generalized Few-Shot Learning
• Predicting Visual Exemplars of Unseen Classes for Zero-Shot Learning
• Learning a Deep Embedding Model for Zero-Shot Learning
• Visually Aligned Word Embeddings for Improving Zero-shot Learning
• Universal Semi-Supervised Semantic Segmentation
• Zero-Shot Transfer Learning for Event Extraction
• CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language
• Large-Scale Visual Relationship Understanding
• Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition
• Generative Adversarial Zero-shot Learning via Knowledge Graphs
• Isometric Propagation Network for Generalized Zero-shot Learning
• SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detection
• Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?
• Learning Structured Semantic Embeddings for Visual Recognition
• Generalized Continual Zero-Shot Learning
• Connecting Touch and Vision via Cross-Modal Prediction
• Generative Replay-based Continual Zero-Shot Learning
• Weak-shot Fine-grained Classification via Similarity Transfer
• Less is more: zero-shot learning from online textual documents with noise suppression
• Zero-Shot Visual Recognition via Bidirectional Latent Embedding
• Zero-shot Learning via Shared-Reconstruction-Graph Pursuit
• Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective
• Simple and effective localized attribute representations for zero-shot learning
• Generalized Zero-Shot Recognition based on Visually Semantic Embedding
• Zero-Shot Semantic Segmentation
• Attribute Propagation Network for Graph Zero-shot Learning
• SR-GAN: Semantic Rectifying Generative Adversarial Network for Zero-shot Learning
• ZstGAN: An Adversarial Approach for Unsupervised Zero-Shot Image-to-Image Translation
• GAN for Vision, KG for Relation: a Two-stage Deep Network for Zero-shot Action Recognition
• Visual Space Optimization for Zero-shot Learning
• An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning
• Towards Effective Deep Embedding for Zero-Shot Learning
• A Large-scale Attribute Dataset for Zero-shot Learning
• Transfer feature generating networks with semantic classes structure for zero-shot learning
• Factors in Finetuning Deep Model for object detection
• Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
• Tell and Predict: Kernel Classifier Prediction for Unseen Visual Classes from Unstructured Text Descriptions
• Zero-Shot Learning with Knowledge Enhanced Visual Semantic Embeddings
• Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification
• Knowledge-aware Zero-Shot Learning: Survey and Perspective
• Adaptive Cross-Modal Few-Shot Learning
• Classifier and Exemplar Synthesis for Zero-Shot Learning
• Unified Generator-Classifier for Efficient Zero-Shot Learning
• Generalized Zero-Shot Learning for Action Recognition with Web-Scale Video Data
• Zero-Shot Learning via Category-Specific Visual-Semantic Mapping
• Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow
• Graceful Degradation and Related Fields
• Class label autoencoder for zero-shot learning
• KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification
• From Fully Supervised to Zero Shot Settings for Twitter Hashtag Recommendation
• SIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation
• Visual-Semantic Embedding Model Informed by Structured Knowledge
• Learning from #Barcelona Instagram data what Locals and Tourists post about its Neighbourhoods
• A Semantics-Guided Class Imbalance Learning Model for Zero-Shot Classification
• Global Semantic Consistency for Zero-Shot Learning
• Joint Dictionaries for Zero-Shot Learning
• Write a Classifier: Predicting Visual Classifiers from Unstructured Text
• From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition
• Compositional Fine-Grained Low-Shot Learning
• Captioning Images with Novel Objects via Online Vocabulary Expansion
• AfriVEC: Word Embedding Models for African Languages. Case Study of Fon and Nobiin
• Context-Aware Zero-Shot Recognition
• Alleviating Feature Confusion for Generative Zero-shot Learning
• Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning
• Infinite-Label Learning with Semantic Output Codes
• Joint Concept Matching based Learning for Zero-Shot Recognition
• Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild
• Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
• Simple multi-dataset detection
• Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability
• Informed Democracy: Voting-based Novelty Detection for Action Recognition
• Hyper-Process Model: A Zero-Shot Learning algorithm for Regression Problems based on Shape Analysis
• AMP0: Species-Specific Prediction of Anti-microbial Peptides using Zero and Few Shot Learning
• Progressive Ensemble Networks for Zero-Shot Recognition
• Learning Class-Transductive Intent Representations for Zero-shot Intent Detection
• Complementary Attributes: A New Clue to Zero-Shot Learning
• Zero-Shot Recognition through Image-Guided Semantic Classification
• Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation
• A Zero-Shot Learning application in Deep Drawing process using Hyper-Process Model
• Learning from Multiple Noisy Partial Labelers
• SEEC: Semantic Vector Federation across Edge Computing Environments
• f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
• CIZSL++: Creativity Inspired Generative Zero-Shot Learning
• Multi-Label Zero-Shot Human Action Recognition via Joint Latent Ranking Embedding
• Learning the Compositional Spaces for Generalized Zero-shot Learning
• Prior Knowledge about Attributes: Learning a More Effective Potential Space for Zero-Shot Recognition
• Neighborhood Sensitive Mapping for Zero-Shot Classification using Independently Learned Semantic Embeddings
• Connecting Context-specific Adaptation in Humans to Meta-learning
• OntoZSL: Ontology-enhanced Zero-shot Learning
• Multimodal Logical Inference System for Visual-Textual Entailment
• Convolutional Prototype Learning for Zero-Shot Recognition
• Unsupervised Open Domain Recognition by Semantic Discrepancy Minimization
• Zero-Shot Learning with Multi-Battery Factor Analysis
• OD-GCN: Object Detection Boosted by Knowledge GCN
• CRL: Class Representative Learning for Image Classification
• Generative Model-driven Structure Aligning Discriminative Embeddings for Transductive Zero-shot Learning
• Representation Quality Of Neural Networks Links To Adversarial Attacks and Defences
• Weak Novel Categories without Tears: A Survey on Weak-Shot Learning
• Attribute-Induced Bias Eliminating for Transductive Zero-Shot Learning
• Improving Generalized Zero-Shot Learning by Semantic Discriminator
• Video Stream Retrieval of Unseen Queries using Semantic Memory
• Opening up Open-World Tracking
• Learning Graph-Based Priors for Generalized Zero-Shot Learning
• LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes
• External-Memory Networks for Low-Shot Learning of Targets in Forward-Looking-Sonar Imagery
• Structure-Aware Feature Generation for Zero-Shot Learning
• ZSTAD: Zero-Shot Temporal Activity Detection
• Zero-Shot Recognition via Optimal Transport
• SDM-Net: A Simple and Effective Model for Generalized Zero-Shot Learning
• A Simple Approach for Zero-Shot Learning based on Triplet Distribution Embeddings
• Revisiting Document Representations for Large-Scale Zero-Shot Learning
• Learning to hash with semantic similarity metrics and empirical KL divergence
• Semantic Graph for Zero-Shot Learning
• Traversing the Continuous Spectrum of Image Retrieval with Deep Dynamic Models
• Zero-Shot Learning from Adversarial Feature Residual to Compact Visual Feature
• Multi-Label Zero-Shot Learning with Transfer-Aware Label Embedding Projection
• A Prototype-Based Generalized Zero-Shot Learning Framework for Hand Gesture Recognition
• Few-Shot Adaptation for Multimedia Semantic Indexing
• Heterogeneous Graph-based Knowledge Transfer for Generalized Zero-shot Learning
• Integrating Propositional and Relational Label Side Information for Hierarchical Zero-Shot Image Classification
• Zero-shot Learning of 3D Point Cloud Objects
• On zero-shot recognition of generic objects
• Discriminative Embedding Autoencoder with a Regressor Feedback for Zero-Shot Learning
• Image2song: Song Retrieval via Bridging Image Content and Lyric Words
• The Artificial Mind's Eye: Resisting Adversarials for Convolutional Neural Networks using Internal Projection