MatFormer: A Generative Model for Procedural Materials

Abstract

Procedural material graphs are a compact, parameteric, and resolution-independent representation that are a popular choice for material authoring. However, designing procedural materials requires significant expertise and publicly accessible libraries contain only a few thousand such graphs. We present MatFormer, a generative model that can produce a diverse set of high-quality procedural materials with complex spatial patterns and appearance. While procedural materials can be modeled as directed (operation) graphs, they contain arbitrary numbers of heterogeneous nodes with unstructured, often long-range node connections, and functional constraints on node parameters and connections. MatFormer addresses these challenges with a multi-stage transformer-based model that sequentially generates nodes, node parameters, and edges, while ensuring the semantic validity of the graph. In addition to generation, MatFormer can be used for the auto-completion and exploration of partial material graphs. We qualitatively and quantitatively demonstrate that our method outperforms alternative approaches, in both generated graph and material quality.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows
• MaterialGAN: Reflectance Capture using a Generative SVBRDF Model
• ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis
• PolyGen: An Autoregressive Generative Model of 3D Meshes
• StructureNet: Hierarchical Graph Networks for 3D Shape Generation
• A Bayesian Inference Framework for Procedural Material Parameter Estimation
• SketchGen: Generating Constrained CAD Sketches

Cited by in corpus (9)

• ControlMat: A Controlled Generative Approach to Material Capture
• MatFusion: A Generative Diffusion Model for SVBRDF Capture
• MatSynth: A Modern PBR Materials Dataset
• PhotoMat: A Material Generator Learned from Single Flash Photos
• Generating Procedural Materials from Text or Image Prompts
• Mesh Neural Cellular Automata
• Editing Motion Graphics Video via Motion Vectorization and Transformation
• GeoCode: Interpretable Shape Programs
• MaterialPicker: Multi-Modal DiT-Based Material Generation