Block Neural Autoregressive Flow

Abstract

Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (B-NAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feed-forward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters.

12 pages, 3 figures, 3 tables

References in corpus (3)

• NICE: Non-linear Independent Components Estimation
• MADE: Masked Autoencoder for Distribution Estimation
• High-Dimensional Probability Estimation with Deep Density Models

Cited by in corpus (9)

• Why Normalizing Flows Fail to Detect Out-of-Distribution Data
• An introduction to variational inference in Geophysical inverse problems
• What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomes
• Neural Density Estimation and Likelihood-free Inference
• DeepSPACE: Approximate Geospatial Query Processing with Deep Learning
• Variational Mixture of Normalizing Flows
• Visual Attention in Imaginative Agents
• A Triangular Network For Density Estimation
• Automatic Guide Generation for Stan via NumPyro