PolyGAN: High-Order Polynomial Generators
arXiv:1908.06571
Abstract
Generative Adversarial Networks (GANs) have become the gold standard when it comes to learning generative models for high-dimensional distributions. Since their advent, numerous variations of GANs have been introduced in the literature, primarily focusing on utilization of novel loss functions, optimization/regularization strategies and network architectures. In this paper, we turn our attention to the generator and investigate the use of high-order polynomials as an alternative class of universal function approximators. Concretely, we propose PolyGAN, where we model the data generator by means of a high-order polynomial whose unknown parameters are naturally represented by high-order tensors. We introduce two tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks that only employ linear/convolutional blocks. We exhibit for the first time that by using our approach a GAN generator can approximate the data distribution without using any activation functions. Thorough experimental evaluation on both synthetic and real data (images and 3D point clouds) demonstrates the merits of PolyGAN against the state of the art.
References in corpus (12)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Generative Adversarial Networks: An Overview
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Spectral Normalization for Generative Adversarial Networks
- Self-Attention Generative Adversarial Networks
- Tensor Decomposition for Signal Processing and Machine Learning
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Learning Functions: When Is Deep Better Than Shallow
- Identity Matters in Deep Learning
- An analytic theory of generalization dynamics and transfer learning in deep linear networks
- Emerging Convolutions for Generative Normalizing Flows