GramGAN: Deep 3D Texture Synthesis From 2D Exemplars
arXiv:2006.16112
Abstract
We present a novel texture synthesis framework, enabling the generation of infinite, high-quality 3D textures given a 2D exemplar image. Inspired by recent advances in natural texture synthesis, we train deep neural models to generate textures by non-linearly combining learned noise frequencies. To achieve a highly realistic output conditioned on an exemplar patch, we propose a novel loss function that combines ideas from both style transfer and generative adversarial networks. In particular, we train the synthesis network to match the Gram matrices of deep features from a discriminator network. In addition, we propose two architectural concepts and an extrapolation strategy that significantly improve generalization performance. In particular, we inject both model input and condition into hidden network layers by learning to scale and bias hidden activations. Quantitative and qualitative evaluations on a diverse set of exemplars motivate our design decisions and show that our system performs superior to previous state of the art. Finally, we conduct a user study that confirms the benefits of our framework.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Conditional Generative Adversarial Nets
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
- Texture Synthesis Using Convolutional Neural Networks
- Texture Synthesis with Spatial Generative Adversarial Networks
- TileGAN: Synthesis of Large-Scale Non-Homogeneous Textures