PixelPyramids: Exact Inference Models from Lossless Image Pyramids

Abstract

Autoregressive models are a class of exact inference approaches with highly flexible functional forms, yielding state-of-the-art density estimates for natural images. Yet, the sequential ordering on the dimensions makes these models computationally expensive and limits their applicability to low-resolution imagery. In this work, we propose Pixel-Pyramids, a block-autoregressive approach employing a lossless pyramid decomposition with scale-specific representations to encode the joint distribution of image pixels. Crucially, it affords a sparser dependency structure compared to fully autoregressive approaches. Our PixelPyramids yield state-of-the-art results for density estimation on various image datasets, especially for high-resolution data. For CelebA-HQ 1024 x 1024, we observe that the density estimates (in terms of bits/dim) are improved to ~44% of the baseline despite sampling speeds superior even to easily parallelizable flow-based models.

arXiv commentTo appear at ICCV 2021

References in corpus (12)

• Auto-Encoding Variational Bayes
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• The challenge of realistic music generation: modelling raw audio at scale
• Generative Image Modeling Using Spatial LSTMs
• Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
• Emerging Convolutions for Generative Normalizing Flows
• Lossless Image Compression through Super-Resolution
• Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving