Regularized Autoencoders via Relaxed Injective Probability Flow

Abstract

Invertible flow-based generative models are an effective method for learning to generate samples, while allowing for tractable likelihood computation and inference. However, the invertibility requirement restricts models to have the same latent dimensionality as the inputs. This imposes significant architectural, memory, and computational costs, making them more challenging to scale than other classes of generative models such as Variational Autoencoders (VAEs). We propose a generative model based on probability flows that does away with the bijectivity requirement on the model and only assumes injectivity. This also provides another perspective on regularized autoencoders (RAEs), with our final objectives resembling RAEs with specific regularizers that are derived by lower bounding the probability flow objective. We empirically demonstrate the promise of the proposed model, improving over VAEs and AEs in terms of sample quality.

AISTATS 2020

References in corpus (16)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Neural Discrete Representation Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Density estimation using Real NVP
• Disentangling by Factorising
• A note on the evaluation of generative models
• Wasserstein Auto-Encoders
• Assessing Generative Models via Precision and Recall
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Diagnosing and Enhancing VAE Models
• Is Generator Conditioning Causally Related to GAN Performance?
• On the Quantitative Analysis of Decoder-Based Generative Models
• On Implicit Regularization in $\beta$-VAEs
• Understanding the (un)interpretability of natural image distributions using generative models
• Variational Noise-Contrastive Estimation

Cited by in corpus (7)

• Probabilistic Autoencoder
• Globally Injective ReLU Networks
• Trumpets: Injective Flows for Inference and Inverse Problems
• Normalizing Flows Across Dimensions
• Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows
• To Regularize or Not To Regularize? The Bias Variance Trade-off in Regularized AEs
• Ordering Dimensions with Nested Dropout Normalizing Flows