LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators

Abstract

Layout is important for graphic design and scene generation. We propose a novel Generative Adversarial Network, called LayoutGAN, that synthesizes layouts by modeling geometric relations of different types of 2D elements. The generator of LayoutGAN takes as input a set of randomly-placed 2D graphic elements and uses self-attention modules to refine their labels and geometric parameters jointly to produce a realistic layout. Accurate alignment is critical for good layouts. We thus propose a novel differentiable wireframe rendering layer that maps the generated layout to a wireframe image, upon which a CNN-based discriminator is used to optimize the layouts in image space. We validate the effectiveness of LayoutGAN in various experiments including MNIST digit generation, document layout generation, clipart abstract scene generation and tangram graphic design.

Accepted as a conference paper at ICLR 2019

Cited by in corpus (12)

• Semantic Object Accuracy for Generative Text-to-Image Synthesis
• SketchGen: Generating Constrained CAD Sketches
• Auto Completion of User Interface Layout Design Using Transformer-Based Tree Decoders
• Neural Design Network: Graphic Layout Generation with Constraints
• Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Models
• Multi-Agent Reinforcement Learning of 3D Furniture Layout Simulation in Indoor Graphics Scenes
• End-to-end Generative Floor-plan and Layout with Attributes and Relation Graph
• Engineering Sketch Generation for Computer-Aided Design
• Attribute-conditioned Layout GAN for Automatic Graphic Design
• Adversarial Model for Rotated Indoor Scenes Planning
• Deep Layout of Custom-size Furniture through Multiple-domain Learning
• MeronymNet: A Hierarchical Approach for Unified and Controllable Multi-Category Object Generation