Image Inspired Poetry Generation in XiaoIce

Abstract

Vision is a common source of inspiration for poetry. The objects and the sentimental imprints that one perceives from an image may lead to various feelings depending on the reader. In this paper, we present a system of poetry generation from images to mimic the process. Given an image, we first extract a few keywords representing objects and sentiments perceived from the image. These keywords are then expanded to related ones based on their associations in human written poems. Finally, verses are generated gradually from the keywords using recurrent neural networks trained on existing poems. Our approach is evaluated by human assessors and compared to other generation baselines. The results show that our method can generate poems that are more artistic than the baseline methods. This is one of the few attempts to generate poetry from images. By deploying our proposed approach, XiaoIce has already generated more than 12 million poems for users since its release in July 2017. A book of its poems has been published by Cheers Publishing, which claimed that the book is the first-ever poetry collection written by an AI in human history.

References in corpus (5)

• Microsoft COCO Captions: Data Collection and Evaluation Server
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Language Models for Image Captioning: The Quirks and What Works
• Chinese Song Iambics Generation with Neural Attention-based Model
• Chinese Poetry Generation with Planning based Neural Network

Cited by in corpus (6)

• Generating lyrics with variational autoencoder and multi-modal artist embeddings
• Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poems
• Shall I Compare Thee to a Machine-Written Sonnet? An Approach to Algorithmic Sonnet Generation
• Generation of lyrics lines conditioned on music audio clips
• Generating Chinese Poetry from Images via Concrete and Abstract Information
• From Knowledge Map to Mind Map: Artificial Imagination