Deep Multimodal Semantic Embeddings for Speech and Images

David Harwath, James Glass

In this paper, we present a model which takes as input a corpus of images with relevant spoken captions and finds a correspondence between the two modalities. We employ a pair of c…