Generative Convolution Layer for Image Generation

Abstract

This paper introduces a novel convolution method, called generative convolution (GConv), which is simple yet effective for improving the generative adversarial network (GAN) performance. Unlike the standard convolution, GConv first selects useful kernels compatible with the given latent vector, and then linearly combines the selected kernels to make latent-specific kernels. Using the latent-specific kernels, the proposed method produces the latent-specific features which encourage the generator to produce high-quality images. This approach is simple but surprisingly effective. First, the GAN performance is significantly improved with a little additional hardware cost. Second, GConv can be employed to the existing state-of-the-art generators without modifying the network architecture. To reveal the superiority of GConv, this paper provides extensive experiments using various standard datasets including CIFAR-10, CIFAR-100, LSUN-Church, CelebA, and tiny-ImageNet. Quantitative evaluations prove that GConv significantly boosts the performances of the unconditional and conditional GANs in terms of Inception score (IS) and Frechet inception distance (FID). For example, the proposed method improves both FID and IS scores on the tiny-ImageNet dataset from 35.13 to 29.76 and 20.23 to 22.64, respectively.

arXiv commentSubmitted to Neural Networks

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Generative Adversarial Text to Image Synthesis
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Language Modeling with Gated Convolutional Networks
• Wasserstein GAN
• cGANs with Projection Discriminator
• On Convergence and Stability of GANs
• Stabilizing Training of Generative Adversarial Networks through Regularization
• PEPSI++: Fast and Lightweight Network for Image Inpainting
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• Simple yet Effective Way for Improving the Performance of GAN
• Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection
• cGANs with Conditional Convolution Layer