Sum-of-Squares Polynomial Flow

Abstract

Triangular map is a recent construct in probability theory that allows one to transform any source probability density function to any target density function. Based on triangular maps, we propose a general framework for high-dimensional density estimation, by specifying one-dimensional transformations (equivalently conditional densities) and appropriate conditioner networks. This framework (a) reveals the commonalities and differences of existing autoregressive and flow based methods, (b) allows a unified understanding of the limitations and representation power of these recent approaches and, (c) motivates us to uncover a new Sum-of-Squares (SOS) flow that is interpretable, universal, and easy to train. We perform several synthetic experiments on various density geometries to demonstrate the benefits (and short-comings) of such transformations. SOS flows achieve competitive results in simulations and several real-world datasets.

arXiv comment13 pages, ICML'2019

Cited by in corpus (24)

• Normalizing Flows: An Introduction and Review of Current Methods
• Normalizing Flows for Probabilistic Modeling and Inference
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows
• Neural Spline Flows
• Intensity-Free Learning of Temporal Point Processes
• Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows
• Wavelet Flow: Fast Training of High Resolution Normalizing Flows
• Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
• Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
• Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
• Fast and Flexible Temporal Point Processes with Triangular Maps
• Sparse Flows: Pruning Continuous-depth Models
• The Convolution Exponential and Generalized Sylvester Flows
• Closing the Dequantization Gap: PixelCNN as a Single-Layer Flow
• Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows
• Density Deconvolution with Normalizing Flows
• Universal Approximation of Residual Flows in Maximum Mean Discrepancy
• Manifold Density Estimation via Generalized Dequantization
• Generalization of the Change of Variables Formula with Applications to Residual Flows
• Improving the expressiveness of neural vocoding with non-affine Normalizing Flows
• Gradient Boosted Normalizing Flows
• Efficient sampling generation from explicit densities via Normalizing Flows
• Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regression