Combining Language and Vision with a Multimodal Skip-gram Model

Angeliki Lazaridou, Nghia The Pham, Marco Baroni

We extend the SKIP-GRAM model of Mikolov et al. (2013a) by taking visual information into account. Like SKIP-GRAM, our multimodal models (MMSKIP-GRAM) build vector-based word repre…