Generative Design through Quality-Diversity Data Synthesis and Language Models

Abstract

Two fundamental challenges face generative models in engineering applications: the acquisition of high-performing, diverse datasets, and the adherence to precise constraints in generated designs. We propose a novel approach combining optimization, constraint satisfaction, and language models to tackle these challenges in architectural design. Our method uses Quality-Diversity (QD) to generate a diverse, high-performing dataset. We then fine-tune a language model with this dataset to generate high-level designs. These designs are then refined into detailed, constraint-compliant layouts using the Wave Function Collapse algorithm. Our system demonstrates reliable adherence to textual guidance, enabling the generation of layouts with targeted architectural and performance features. Crucially, our results indicate that data synthesized through the evolutionary search of QD not only improves overall model performance but is essential for the model's ability to closely adhere to textual guidance. This improvement underscores the pivotal role evolutionary computation can play in creating the datasets key to training generative models for design. Web article at https://tilegpt.github.io

arXiv comment8 pages, 8 figures, GECCO 2024

References in corpus (8)

• Procedural Content Generation through Quality Diversity
• Level Generation Through Large Language Models
• Empowering Quality Diversity in Dungeon Design with Interactive Constrained MAP-Elites
• Discovering Representations for Black-box Optimization
• Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution
• Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuilding
• Designing Air Flow with Surrogate-assisted Phenotypic Niching
• ARCH-Elites: Quality-Diversity for Urban Design