A Multi-Modal Chinese Poetry Generation Model

Abstract

Recent studies in sequence-to-sequence learning demonstrate that RNN encoder-decoder structure can successfully generate Chinese poetry. However, existing methods can only generate poetry with a given first line or user's intent theme. In this paper, we proposed a three-stage multi-modal Chinese poetry generation approach. Given a picture, the first line, the title and the other lines of the poem are successively generated in three stages. According to the characteristics of Chinese poems, we propose a hierarchy-attention seq2seq model which can effectively capture character, phrase, and sentence information between contexts and improve the symmetry delivered in poems. In addition, the Latent Dirichlet allocation (LDA) model is utilized for title generation and improve the relevance of the whole poem and the title. Compared with strong baseline, the experimental results demonstrate the effectiveness of our approach, using machine evaluations as well as human judgments.

arXiv commentAccepted at the International Joint Conference on Neural Networks, IJCNN, 2018

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Distributed Representations of Words and Phrases and their Compositionality
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• Variational Recurrent Auto-Encoders
• Attribute2Image: Conditional Image Generation from Visual Attributes
• Chinese Song Iambics Generation with Neural Attention-based Model
• Chinese Poetry Generation with Planning based Neural Network
• Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation
• Backward and Forward Language Modeling for Constrained Sentence Generation
• Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test

Cited by in corpus (6)

• When Creators Meet the Metaverse: A Survey on Computational Arts
• Deep Poetry: A Chinese Classical Poetry Generation System
• Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poems
• AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation
• Generating Chinese Poetry from Images via Concrete and Abstract Information
• WakaVT: A Sequential Variational Transformer for Waka Generation