Neural Spline Flows

Abstract

A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choice determines the flexibility of these models. Building upon recent work, we propose a fully-differentiable module based on monotonic rational-quadratic splines, which enhances the flexibility of both coupling and autoregressive transforms while retaining analytic invertibility. We demonstrate that neural spline flows improve density estimation, variational inference, and generative modeling of images.

Published at the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

References in corpus (22)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• WaveNet: A Generative Model for Raw Audio
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• EMNIST: an extension of MNIST to handwritten letters
• MADE: Masked Autoencoder for Distribution Estimation
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• RNADE: The real-valued neural autoregressive density-estimator
• Improving Variational Auto-Encoders using Householder Flow
• Latent Normalizing Flows for Discrete Sequences
• Sum-of-Squares Polynomial Flow
• Taming VAEs
• FloWaveNet : A Generative Flow for Raw Audio
• Cubic-Spline Flows
• Reversible Recurrent Neural Networks
• Autoregressive Energy Machines
• Deep Diffeomorphic Normalizing Flows

Cited by in corpus (20)

• Anomaly Detection with Density Estimation
• Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC
• Equivariant Flows: sampling configurations for multi-body systems with symmetric energies
• IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
• Bayesian Stokes inversion with Normalizing flows
• Neural Approximate Sufficient Statistics for Implicit Models
• NFAD: Fixing anomaly detection using normalizing flows
• Deep Potential: Recovering the gravitational potential from a snapshot of phase space
• Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions
• Trumpets: Injective Flows for Inference and Inverse Problems
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Sparse Flows: Pruning Continuous-depth Models
• Identifying through Flows for Recovering Latent Representations
• Robust normalizing flows using Bernstein-type polynomials
• Woodbury Transformations for Deep Generative Flows
• FIS-GAN: GAN with Flow-based Importance Sampling
• Resampling Base Distributions of Normalizing Flows
• Optimal Design of Experiments for Simulation-Based Inference of Mechanistic Acyclic Biological Networks
• CInC Flow: Characterizable Invertible 3x3 Convolution
• Improving the expressiveness of neural vocoding with non-affine Normalizing Flows