Label-Embedding for Image Classification

Abstract

Attributes act as intermediate representations that enable parameter sharing between classes, a must when training data is scarce. We propose to view attribute-based image classification as a label-embedding problem: each class is embedded in the space of attribute vectors. We introduce a function that measures the compatibility between an image and a label embedding. The parameters of this function are learned on a training set of labeled samples to ensure that, given an image, the correct classes rank higher than the incorrect ones. Results on the Animals With Attributes and Caltech-UCSD-Birds datasets show that the proposed framework outperforms the standard Direct Attribute Prediction baseline in a zero-shot learning scenario. Label embedding enjoys a built-in ability to leverage alternative sources of information instead of or in addition to attributes, such as e.g. class hierarchies or textual descriptions. Moreover, label embedding encompasses the whole range of learning settings from zero-shot learning to regular learning with a large number of labeled examples.

IEEE TPAMI preprint

References in corpus (3)

• Zero-Shot Learning Through Cross-Modal Transfer
• Evaluation of Output Embeddings for Fine-Grained Image Classification
• Multi-Label Prediction via Compressed Sensing

Cited by in corpus (94)

• Beyond One-hot Encoding: lower dimensional target embedding
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning
• Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
• Learning Deep Representations of Fine-grained Visual Descriptions
• Context-aware Feature Generation for Zero-shot Semantic Segmentation
• Zero-Shot Detection
• Leveraging the Invariant Side of Generative Zero-Shot Learning
• Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning
• A Transfer Learning Evaluation of Deep Neural Networks for Image Classification
• Zero-Shot Learning and its Applications from Autonomous Vehicles to COVID-19 Diagnosis: A Review
• Zero-Shot Learning by Generating Pseudo Feature Representations
• Transductive Zero-Shot Learning with Visual Structure Constraint
• Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classification
• Disentangling Semantic-to-visual Confusion for Zero-shot Learning
• Vocabulary-informed Zero-shot and Open-set Learning
• Rebalanced Zero-shot Learning
• Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learning
• Learning Action Representations for Reinforcement Learning
• Dual Progressive Prototype Network for Generalized Zero-Shot Learning
• Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objects
• Fine-Grained Zero-Shot Learning with DNA as Side Information
• Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records
• On Implicit Attribute Localization for Generalized Zero-Shot Learning
• Online Lifelong Generalized Zero-Shot Learning
• Relational Generalized Few-Shot Learning
• From Classical to Generalized Zero-Shot Learning: a Simple Adaptation Process
• CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language
• FREE: Feature Refinement for Generalized Zero-Shot Learning
• Isometric Propagation Network for Generalized Zero-shot Learning
• Generative Replay-based Continual Zero-Shot Learning
• Generalized Continual Zero-Shot Learning
• Learning to Recommend Signal Plans under Incidents with Real-Time Traffic Prediction
• Simple and effective localized attribute representations for zero-shot learning
• Zero-Shot Semantic Segmentation
• SR-GAN: Semantic Rectifying Generative Adversarial Network for Zero-shot Learning
• Cross-modal Cognitive Consensus guided Audio-Visual Segmentation
• Continuous Conditional Generative Adversarial Networks: Novel Empirical Losses and Label Input Mechanisms
• Attribute Propagation Network for Graph Zero-shot Learning
• Region Semantically Aligned Network for Zero-Shot Learning
• An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning
• A Boundary Based Out-of-Distribution Classifier for Generalized Zero-Shot Learning
• Visual Space Optimization for Zero-shot Learning
• Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
• Transfer feature generating networks with semantic classes structure for zero-shot learning
• Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification
• Zero-Shot Learning with Knowledge Enhanced Visual Semantic Embeddings
• Hardness Sampling for Self-Training Based Transductive Zero-Shot Learning
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning
• Structure propagation for zero-shot learning
• Zero-Shot Audio Classification via Semantic Embeddings
• Elaborative Rehearsal for Zero-shot Action Recognition
• Bookworm continual learning: beyond zero-shot learning and continual learning
• Exploring Meta Information for Audio-based Zero-shot Bird Classification
• Alleviating Feature Confusion for Generative Zero-shot Learning
• A Semantics-Guided Class Imbalance Learning Model for Zero-Shot Classification
• SIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation
• Hierarchical Prototype Learning for Zero-Shot Recognition
• Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches
• Domain-Specific Embedding Network for Zero-Shot Recognition
• Complementary Attributes: A New Clue to Zero-Shot Learning
• Zero-Shot Audio Classification with Factored Linear and Nonlinear Acoustic-Semantic Projections
• Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning
• Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
• Recursive Training for Zero-Shot Semantic Segmentation
• Hyper-Process Model: A Zero-Shot Learning algorithm for Regression Problems based on Shape Analysis
• Bi-Semantic Reconstructing Generative Network for Zero-shot Learning
• Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation
• Improving Generalized Zero-Shot Learning by Semantic Discriminator
• Semi-supervised Learning with a Teacher-student Network for Generalized Attribute Prediction
• Angle-based hierarchical classification using exact label embedding
• CIZSL++: Creativity Inspired Generative Zero-Shot Learning
• Learning Classifiers for Domain Adaptation, Zero and Few-Shot Recognition Based on Learning Latent Semantic Parts
• Learning the Compositional Spaces for Generalized Zero-shot Learning
• A Zero-Shot Learning application in Deep Drawing process using Hyper-Process Model
• CRL: Class Representative Learning for Image Classification
• AlphaNet: Improving Long-Tail Classification By Combining Classifiers
• Discriminative Embedding Autoencoder with a Regressor Feedback for Zero-Shot Learning
• Revisiting Document Representations for Large-Scale Zero-Shot Learning
• A Simple Approach for Zero-Shot Learning based on Triplet Distribution Embeddings
• Learning Single/Multi-Attribute of Object with Symmetry and Group
• Universal Multi-Source Domain Adaptation
• LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes
• Integrating Propositional and Relational Label Side Information for Hierarchical Zero-Shot Image Classification
• MUTE: Data-Similarity Driven Multi-hot Target Encoding for Neural Network Design
• Heterogeneous Graph-based Knowledge Transfer for Generalized Zero-shot Learning
• Mitigating Generation Shifts for Generalized Zero-Shot Learning
• Developing Motion Code Embedding for Action Recognition in Videos
• SDM-Net: A Simple and Effective Model for Generalized Zero-Shot Learning
• Transductive Zero-Shot Learning by Decoupled Feature Generation
• A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion
• Toward Zero-Shot Unsupervised Image-to-Image Translation
• Improving Generalization via Attribute Selection on Out-of-the-box Data