Transductive Multi-view Zero-Shot Learning

Abstract

Most existing zero-shot learning approaches exploit transfer learning via an intermediate-level semantic representation shared between an annotated auxiliary dataset and a target dataset with different classes and no annotation. A projection from a low-level feature space to the semantic representation space is learned from the auxiliary dataset and is applied without adaptation to the target dataset. In this paper we identify two inherent limitations with these approaches. First, due to having disjoint and potentially unrelated classes, the projection functions learned from the auxiliary dataset/domain are biased when applied directly to the target dataset/domain. We call this problem the projection domain shift problem and propose a novel framework, transductive multi-view embedding, to solve it. The second limitation is the prototype sparsity problem which refers to the fact that for each target class, only a single prototype is available for zero-shot learning given a semantic representation. To overcome this problem, a novel heterogeneous multi-view hypergraph label propagation method is formulated for zero-shot learning in the transductive embedding space. It effectively exploits the complementary information offered by different semantic representations and takes advantage of the manifold structures of multiple representation spaces in a coherent manner. We demonstrate through extensive experiments that the proposed approach (1) rectifies the projection shift between the auxiliary and target domains, (2) exploits the complementarity of multiple semantic representations, (3) significantly outperforms existing methods for both zero-shot and N-shot recognition on three image and video benchmark datasets, and (4) enables novel cross-view annotation tasks.

accepted by IEEE TPAMI, more info and longer report will be available in :http://www.eecs.qmul.ac.uk/~yf300/embedding/index.html

References in corpus (7)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Zero-Shot Learning Through Cross-Modal Transfer
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• A PAC-Bayesian bound for Lifelong Learning
• Context-Aware Hypergraph Construction for Robust Spectral Clustering
• Contextual Hypergraph Modelling for Salient Object Detection
• A Multi-View Embedding Space for Modeling Internet Images, Tags, and their Semantics

Cited by in corpus (88)

• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• A Review of Generalized Zero-Shot Learning Methods
• Multi-level Semantic Feature Augmentation for One-shot Learning
• Learning Deep Representations of Fine-grained Visual Descriptions
• Semantic Autoencoder for Zero-Shot Learning
• Semantics-Guided Contrastive Network for Zero-Shot Object detection
• Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation
• Context-aware Feature Generation for Zero-shot Semantic Segmentation
• Generalized Multi-view Embedding for Visual Recognition and Cross-modal Retrieval
• Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization
• An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
• Synthesized Classifiers for Zero-Shot Learning
• Zero-Shot Learning and its Applications from Autonomous Vehicles to COVID-19 Diagnosis: A Review
• OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
• Zero-Shot Learning via Class-Conditioned Deep Generative Models
• Zero-Shot Learning by Generating Pseudo Feature Representations
• Transductive Zero-Shot Learning with Visual Structure Constraint
• Domain Adaptation Meets Zero-Shot Learning: An Annotation-Efficient Approach to Multi-Modality Medical Image Segmentation
• From Zero-shot Learning to Conventional Supervised Classification: Unseen Visual Data Synthesis
• Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
• Zero-Shot Learning posed as a Missing Data Problem
• Discriminative Learning of Latent Features for Zero-Shot Recognition
• Semi-supervised Vocabulary-informed Learning
• Stacked Semantic-Guided Attention Model for Fine-Grained Zero-Shot Learning
• Graph-based Visual-Semantic Entanglement Network for Zero-shot Image Recognition
• Vocabulary-informed Zero-shot and Open-set Learning
• Transferable Contrastive Network for Generalized Zero-Shot Learning
• Bias-Eliminated Semantic Refinement for Any-Shot Learning
• Contrastive Embedding for Generalized Zero-Shot Learning
• Semantically Consistent Regularization for Zero-Shot Recognition
• Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks
• Transductive Zero-Shot Hashing for Multilabel Image Retrieval
• Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition
• Learning Joint Feature Adaptation for Zero-Shot Recognition
• From Classical to Generalized Zero-Shot Learning: a Simple Adaptation Process
• Learning a Deep Embedding Model for Zero-Shot Learning
• Visual-Semantic Graph Matching Net for Zero-Shot Learning
• Generative Adversarial Zero-shot Learning via Knowledge Graphs
• Deep Matching Autoencoders
• Zero-Shot Visual Recognition via Bidirectional Latent Embedding
• Query Attack via Opposite-Direction Feature:Towards Robust Image Retrieval
• Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation
• Visual Space Optimization for Zero-shot Learning
• Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention
• Region Semantically Aligned Network for Zero-Shot Learning
• Learning to Infer Unseen Single-/Multi-Attribute-Object Compositions with Graph Networks
• Zero-Shot Crowd Behavior Recognition
• MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning
• Hybrid-Attention based Decoupled Metric Learning for Zero-Shot Image Retrieval
• A Large-scale Attribute Dataset for Zero-shot Learning
• Transfer feature generating networks with semantic classes structure for zero-shot learning
• Multi-Task Curriculum Transfer Deep Learning of Clothing Attributes
• Classifier and Exemplar Synthesis for Zero-Shot Learning
• Hardness Sampling for Self-Training Based Transductive Zero-Shot Learning
• Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
• Vocabulary-informed Extreme Value Learning
• Energy Confused Adversarial Metric Learning for Zero-Shot Image Retrieval and Clustering
• Multi-Knowledge Fusion for New Feature Generation in Generalized Zero-Shot Learning
• Structure propagation for zero-shot learning
• SEER-ZSL: Semantic Encoder-Enhanced Representations for Generalized Zero-Shot Learning
• Sampled in Pairs and Driven by Text: A New Graph Embedding Framework
• ATZSL: Defensive Zero-Shot Recognition in the Presence of Adversaries
• Compositional Fine-Grained Low-Shot Learning
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Zero-Shot Learning via Latent Space Encoding
• A Zero-Shot based Fingerprint Presentation Attack Detection System
• Rapid COVID-19 Risk Screening by Eye-region Manifestations
• Transductive Zero-Shot Hashing via Coarse-to-Fine Similarity Mining
• Learning Class-Transductive Intent Representations for Zero-shot Intent Detection
• Progressive Ensemble Networks for Zero-Shot Recognition
• Zero-sample surface defect detection and classification based on semantic feedback neural network
• Class Knowledge Overlay to Visual Feature Learning for Zero-Shot Image Classification
• Zero-Shot Learning with Multi-Battery Factor Analysis
• Learning the Compositional Spaces for Generalized Zero-shot Learning
• OntoZSL: Ontology-enhanced Zero-shot Learning
• Prior Knowledge about Attributes: Learning a More Effective Potential Space for Zero-Shot Recognition
• Sketch-a-Classifier: Sketch-based Photo Classifier Generation
• Convolutional Prototype Learning for Zero-Shot Recognition
• f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
• Representation Learning for Words and Entities
• Will It Zero-Shot?: Predicting Zero-Shot Classification Performance For Arbitrary Queries
• Multi-Label Zero-Shot Learning with Transfer-Aware Label Embedding Projection
• Zero-Shot Recognition via Optimal Transport
• Node Attribute Generation on Graphs
• Transductive Zero-Shot Learning using Cross-Modal CycleGAN
• Zero-Shot Learning with Sparse Attribute Propagation
• Zero-Shot Learning Based on Knowledge Sharing
• Adaptive Adjustment with Semantic Feature Space for Zero-Shot Recognition