Quasi-Autoregressive Residual (QuAR) Flows
arXiv:2009.07419
Abstract
Normalizing Flows are a powerful technique for learning and modeling probability distributions given samples from those distributions. The current state of the art results are built upon residual flows as these can model a larger hypothesis space than coupling layers. However, residual flows are extremely computationally expensive both to train and to use, which limits their applicability in practice. In this paper, we introduce a simplification to residual flows using a Quasi-Autoregressive (QuAR) approach. Compared to the standard residual flow approach, this simplification retains many of the benefits of residual flows while dramatically reducing the compute time and memory requirements, thus making flow-based modeling approaches far more tractable and broadening their potential applicability.
arXiv commentAppeared in ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models 2020
References in corpus (7)
- Spectral Normalization for Generative Adversarial Networks
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- NICE: Non-linear Independent Components Estimation
- Density estimation using Real NVP
- Neural Ordinary Differential Equations
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Sylvester Normalizing Flows for Variational Inference