DALLE-URBAN: Capturing the urban design expertise of large text to image transformers
arXiv:2208.04139 · doi:10.1109/DICTA56598.2022.10034603
Abstract
Automatically converting text descriptions into images using transformer architectures has recently received considerable attention. Such advances have implications for many applied design disciplines across fashion, art, architecture, urban planning, landscape design and the future tools available to such disciplines. However, a detailed analysis capturing the capabilities of such models, specifically with a focus on the built environment, has not been performed to date. In this work, we investigate the capabilities and biases of such text-to-image methods as it applies to the built environment in detail. We use a systematic grammar to generate queries related to the built environment and evaluate resulting generated images. We generate 1020 different images and find that text to image transformers are robust at generating realistic images across different domains for this use-case. Generated imagery can be found at the github: https://github.com/sachith500/DALLEURBAN
Accepted to DICTA 2022, released 11000+ environmental scene images generated by Stable Diffusion and 1000+ images generated by DALLE-2
References in corpus (13)
- Hierarchical Text-Conditional Image Generation with CLIP Latents
- Diffusion Models Beat GANs on Image Synthesis
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
- DreamFusion: Text-to-3D using 2D Diffusion
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
- Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
- Make-A-Video: Text-to-Video Generation without Text-Video Data
- DiGress: Discrete Denoising diffusion for graph generation
- Streetscape augmentation using generative adversarial networks: insights related to health and wellbeing
- On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
- Discovering the Hidden Vocabulary of DALLE-2
- Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations