An Empirical Study of Language CNN for Image Captioning
arXiv:1612.07086
Abstract
Language Models based on recurrent neural networks have dominated recent image caption generation tasks. In this paper, we introduce a Language CNN model which is suitable for statistical language modeling tasks and shows competitive performance in image captioning. In contrast to previous models which predict next word based on one previous word and hidden state, our language CNN is fed with all the previous words and can model the long-range dependencies of history words, which are critical for image captioning. The effectiveness of our approach is validated on two datasets MS COCO and Flickr30K. Our extensive experimental results show that our method outperforms the vanilla recurrent neural network based language models and is competitive with the state-of-the-art methods.
Comments: 10 pages, In proceedings of ICCV 2017
References in corpus (10)
- Sequence to Sequence Learning with Neural Networks
- Convolutional Neural Network Architectures for Matching Natural Language Sentences
- Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
- A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
- Variational Autoencoder for Deep Learning of Images, Labels and Captions
- Attention Correctness in Neural Image Captioning
- Recurrent Attentional Networks for Saliency Detection
- Phrase-based Image Captioning
- Learning Attributes Equals Multi-Source Domain Generalization
- The Statistical Recurrent Unit