GILT: Generating Images from Long Text

Abstract

Creating an image reflecting the content of a long text is a complex process that requires a sense of creativity. For example, creating a book cover or a movie poster based on their summary or a food image based on its recipe. In this paper we present the new task of generating images from long text that does not describe the visual content of the image directly. For this, we build a system for generating high-resolution 256 $\times$ 256 images of food conditioned on their recipes. The relation between the recipe text (without its title) to the visual content of the image is vague, and the textual structure of recipes is complex, consisting of two sections (ingredients and instructions) both containing multiple sentences. We used the recipe1M dataset to train and evaluate our model that is based on a the StackGAN-v2 architecture.

References in corpus (5)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Improved Techniques for Training GANs
• A Note on the Inception Score

Cited by in corpus (5)

• Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation
• MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
• New Ideas and Trends in Deep Multimodal Content Understanding: A Review
• CookGAN: Meal Image Synthesis from Ingredients
• Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN