Generative Models as Distributions of Functions

Abstract

Generative models are typically trained on grid-like data such as images. As a result, the size of these models usually scales directly with the underlying grid resolution. In this paper, we abandon discretized grids and instead parameterize individual data points by continuous functions. We then build generative models by learning distributions over such functions. By treating data points as functions, we can abstract away from the specific type of data we train on and construct models that are agnostic to discretization. To train our model, we use an adversarial approach with a discriminator that acts on continuous signals. Through experiments on a wide variety of data modalities including images, 3D shapes and climate data, we demonstrate that our model can learn rich distributions of functions independently of data type and resolution.

AISTATS 2022 Oral camera ready. Incorporated reviewer feedback

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Accelerating 3D Deep Learning with PyTorch3D
• Learning Continuous Image Representation with Local Implicit Image Function
• CKConv: Continuous Kernel Convolution For Sequential Data

Cited by in corpus (9)

• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• COIN: COmpression with Implicit Neural representations
• CKConv: Continuous Kernel Convolution For Sequential Data
• Improved Transformer for High-Resolution GANs
• Learning Partial Differential Equations in Reproducing Kernel Hilbert Spaces
• HyperCube: Implicit Field Representations of Voxelized 3D Models
• Learning Signal-Agnostic Manifolds of Neural Fields
• Bayesian Imaging With Data-Driven Priors Encoded by Neural Networks: Theory, Methods, and Algorithms
• Towards Lightweight Controllable Audio Synthesis with Conditional Implicit Neural Representations