STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset

Abstract

In recent years, automatic generation of image descriptions (captions), that is, image captioning, has attracted a great deal of attention. In this paper, we particularly consider generating Japanese captions for images. Since most available caption datasets have been constructed for English language, there are few datasets for Japanese. To tackle this problem, we construct a large-scale Japanese image caption dataset based on images from MS-COCO, which is called STAIR Captions. STAIR Captions consists of 820,310 Japanese captions for 164,062 images. In the experiment, we show that a neural network trained using STAIR Captions can generate more natural and better Japanese captions, compared to those generated using English-Japanese machine translation after generating English captions.

arXiv commentAccepted as ACL2017 short paper. 5 pages

Cited by in corpus (7)

• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• Emergent Translation in Multi-Agent Communication
• M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training
• Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques
• Towards Zero-shot Cross-lingual Image Retrieval and Tagging
• A Corpus for English-Japanese Multimodal Neural Machine Translation with Comparable Sentences
• Models of Visually Grounded Speech Signal Pay Attention To Nouns: a Bilingual Experiment on English and Japanese