9 papers · 1 filter
Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signals
Shah Nawaz, Muhammad Kamran Janjua, Ignazio Gallo +2
We propose a novel deep training algorithm for joint representation of audio and visual information which consists of a single stream network (SSNet) coupled with a novel loss func…
Picture What you Read
Ignazio Gallo, Shah Nawaz, Alessandro Calefati +2
Visualization refers to our ability to create an image in our head based on the text we read or the words we hear. It is one of the many skills that makes reading comprehension pos…
Do Cross Modal Systems Leverage Semantic Relationships?
Shah Nawaz, Muhammad Kamran Janjua, Ignazio Gallo +3
Current cross-modal retrieval systems are evaluated using R@K measure which does not leverage semantic relationships rather strictly follows the manually marked image text query pa…
Aiding Intra-Text Representations with Visual Context for Multimodal Named Entity Recognition
Omer Arshad, Ignazio Gallo, Shah Nawaz +1
With massive explosion of social media such as Twitter and Instagram, people daily share billions of multimedia posts, containing images and text. Typically, text in these posts is…
Learning Inward Scaled Hypersphere Embedding: Exploring Projections in Higher Dimensions
Muhammad Kamran Janjua, Shah Nawaz, Alessandro Calefati +1
Majority of the current dimensionality reduction or retrieval techniques rely on embedding the learned feature representations onto a computable metric space. Once the learned feat…
Image and Encoded Text Fusion for Multi-Modal Classification
Ignazio Gallo, Alessandro Calefati, Shah Nawaz +1
Multi-modal approaches employ data from multiple input streams such as textual and visual domains. Deep neural networks have been successfully employed for these approaches. In thi…