7.4k citations · 11.1k across the 11 of their papers we have counts for
8 papers · 1 filter
Order-Embeddings of Images and Language
Ivan Vendrov, Ryan Kiros, Sanja Fidler +1
Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images. In this paper we advocate…
Action Recognition using Visual Attention
Shikhar Sharma, Ryan Kiros, Ruslan Salakhutdinov
We propose a soft attention based model for the task of action recognition in videos. We use multi-layered Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units…
Exploring Models and Data for Image Question Answering
Mengye Ren, Ryan Kiros, Richard Zemel
This work aims to address the problem of image-based question-answering (QA) with new models and datasets. In our work, we propose to use neural networks and visual semantic embedd…
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Kelvin Xu, Jimmy Ba, Ryan Kiros +5
Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe…
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Ryan Kiros, Ruslan Salakhutdinov, Richard S. Zemel
Inspired by recent advances in multimodal learning and machine translation, we introduce an encoder-decoder pipeline that learns (a): a multimodal joint embedding space with images…
A Multiplicative Model for Learning Distributed Text-Based Attribute Representations
Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov
In this paper we propose a general framework for learning distributed representations of attributes: characteristics of text whose representations can be jointly learned with word…