Unsupervised Stylish Image Description Generation via Domain Layer Norm
arXiv:1809.06214
Abstract
Most of the existing works on image description focus on generating expressive descriptions. The only few works that are dedicated to generating stylish (e.g., romantic, lyric, etc.) descriptions suffer from limited style variation and content digression. To address these limitations, we propose a controllable stylish image description generation model. It can learn to generate stylish image descriptions that are more related to image content and can be trained with the arbitrary monolingual corpus without collecting new paired image and stylish descriptions. Moreover, it enables users to generate various stylish descriptions by plugging in style-specific parameters to include new styles into the existing model. We achieve this capability via a novel layer normalization layer design, which we will refer to as the Domain Layer Norm (DLN). Extensive experimental validation and user study on various stylish image description generation tasks are conducted to show the competitive advantages of the proposed model.
References in corpus (12)
- Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
- A Learned Representation For Artistic Style
- Style Transfer from Non-Parallel Text by Cross-Alignment
- Unsupervised Image-to-Image Translation Networks
- Layer Normalization
- Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
- Towards Diverse and Natural Image Descriptions via a Conditional GAN
- Contrastive Learning for Image Captioning
- Recurrent Topic-Transition GAN for Visual Paragraph Generation
- Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
- SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text
- Improved Neural Text Attribute Transfer with Non-parallel Data