Sylvester Normalizing Flows for Variational Inference

Abstract

Variational inference relies on flexible approximate posterior distributions. Normalizing flows provide a general recipe to construct flexible variational posteriors. We introduce Sylvester normalizing flows, which can be seen as a generalization of planar flows. Sylvester normalizing flows remove the well-known single-unit bottleneck from planar flows, making a single transformation much more flexible. We compare the performance of Sylvester normalizing flows against planar flows and inverse autoregressive flows and demonstrate that they compare favorably on several datasets.

arXiv commentPublished at UAI 2018, 12 pages, 3 figures, code at: https://github.com/riannevdberg/sylvester-flows

References in corpus (3)

• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Density estimation using Real NVP
• Improving Variational Auto-Encoders using Householder Flow

Cited by in corpus (52)

• An Introduction to Variational Autoencoders
• Normalizing Flows: An Introduction and Review of Current Methods
• Neural Ordinary Differential Equations
• FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
• Roundtrip: A Deep Generative Neural Density Estimator
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
• Sorting out Lipschitz function approximation
• Generalized Variational Inference: Three arguments for deriving new Posteriors
• WaveFlow: A Compact Flow-based Model for Raw Audio
• Challenges for Unsupervised Anomaly Detection in Particle Physics
• Improving Variational Autoencoders for New Physics Detection at the LHC with Normalizing Flows
• Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming
• Trivializations for Gradient-Based Optimization on Manifolds
• A Tutorial on Deep Latent Variable Models of Natural Language
• Model-agnostic search for dijet resonances with anomalous jet substructure in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
• Semi-Conditional Normalizing Flows for Semi-Supervised Learning
• Self-Supervised Variational Auto-Encoders
• Generative Latent Flow
• Stochastic Normalizing Flows
• Unsupervised Out-of-Distribution Detection with Batch Normalization
• TraDE: Transformers for Density Estimation
• Graphical Normalizing Flows
• MintNet: Building Invertible Neural Networks with Masked Convolutions
• Deep Diffeomorphic Normalizing Flows
• Invertible DenseNets with Concatenated LipSwish
• On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder
• Super-resolution Variational Auto-Encoders
• Stochastic Neural Network with Kronecker Flow
• Variational Autoencoders for Generative Modelling of Water Cherenkov Detectors
• Hamiltonian Variational Auto-Encoder
• Max-Affine Spline Insights into Deep Generative Networks
• Learning normalizing flows from Entropy-Kantorovich potentials
• Sparse Flows: Pruning Continuous-depth Models
• Backpropagation for Implicit Spectral Densities
• Robust normalizing flows using Bernstein-type polynomials
• Hypernetwork approach to generating point clouds
• MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining
• Woodbury Transformations for Deep Generative Flows
• Structured Dropout Variational Inference for Bayesian Neural Networks
• Quasi-Autoregressive Residual (QuAR) Flows
• Augmented KRnet for density estimation and approximation
• Cauchy-Schwarz Regularized Autoencoder
• Scaling Bayesian inference of mixed multinomial logit models to very large datasets
• General Invertible Transformations for Flow-based Generative Modeling
• Copula-like Variational Inference
• Semi-supervised Grasp Detection by Representation Learning in a Vector Quantized Latent Space
• High Mutual Information in Representation Learning with Symmetric Variational Inference
• TzK: Flow-Based Conditional Generative Model
• Learning more expressive joint distributions in multimodal variational methods
• Gradient Boosted Normalizing Flows