Generalized Multi-view Embedding for Visual Recognition and Cross-modal Retrieval

Abstract

In this paper, the problem of multi-view embedding from different visual cues and modalities is considered. We propose a unified solution for subspace learning methods using the Rayleigh quotient, which is extensible for multiple views, supervised learning, and non-linear embeddings. Numerous methods including Canonical Correlation Analysis, Partial Least Sqaure regression and Linear Discriminant Analysis are studied using specific intrinsic and penalty graphs within the same framework. Non-linear extensions based on kernels and (deep) neural networks are derived, achieving better performance than the linear ones. Moreover, a novel Multi-view Modular Discriminant Analysis (MvMDA) is proposed by taking the view difference into consideration. We demonstrate the effectiveness of the proposed multi-view embedding methods on visual object recognition and cross-modal image retrieval, and obtain superior results in both applications compared to related methods.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distributed Representations of Words and Phrases and their Compositionality
• PCANet: A Simple Deep Learning Baseline for Image Classification?
• A Survey on Multi-view Learning
• Skip-Thought Vectors
• Transductive Multi-view Zero-Shot Learning
• Cross-Modal Learning via Pairwise Constraints

Cited by in corpus (13)

• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Multimodal Subspace Support Vector Data Description
• Supervised Domain Adaptation: A Graph Embedding Perspective and a Rectified Experimental Protocol
• Speed-up and multi-view extensions to Subclass Discriminant Analysis
• Multi-View Class Incremental Learning
• Deep Multi-view Learning to Rank
• Learning Multi-Modal Nonlinear Embeddings: Performance Bounds and an Algorithm
• Deep Tensor CCA for Multi-view Learning
• Neural Class-Specific Regression for face verification
• Randomized Kernel Multi-view Discriminant Analysis
• Trace Ratio Optimization with an Application to Multi-view Learning
• Multi-view Orthonormalized Partial Least Squares: Regularizations and Deep Extensions
• Consistency-aware and Inconsistency-aware Graph-based Multi-view Clustering