Augmented KRnet for density estimation and approximation
arXiv:2105.12866
Abstract
In this work, we have proposed augmented KRnets including both discrete and continuous models. One difficulty in flow-based generative modeling is to maintain the invertibility of the transport map, which is often a trade-off between effectiveness and robustness. The exact invertibility has been achieved in the real NVP using a specific pattern to exchange information between two separated groups of dimensions. KRnet has been developed to enhance the information exchange among data dimensions by incorporating the Knothe-Rosenblatt rearrangement into the structure of the transport map. Due to the maintenance of exact invertibility, a full nonlinear update of all data dimensions needs three iterations in KRnet. To alleviate this issue, we will add augmented dimensions that act as a channel for communications among the data dimensions. In the augmented KRnet, a fully nonlinear update is achieved in two iterations. We also show that the augmented KRnet can be reformulated as the discretization of a neural ODE, where the exact invertibility is kept such that the adjoint method can be formulated with respect to the discretized ODE to obtain the exact gradient. Numerical experiments have been implemented to demonstrate the effectiveness of our models.
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- Deep Residual Learning for Image Recognition
- NICE: Non-linear Independent Components Estimation
- Variational Inference with Normalizing Flows
- Density estimation using Real NVP
- Neural Ordinary Differential Equations
- Masked Autoregressive Flow for Density Estimation
- Sylvester Normalizing Flows for Variational Inference
- How to train your neural ODE: the world of Jacobian and kinetic regularization
- Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
- Augmented Neural ODEs
- Adaptive deep density approximation for Fokker-Planck equations
- Monge-Ampère Flow for Generative Modeling
- Inference via low-dimensional couplings
- Coupling the reduced-order model and the generative model for an importance sampling estimator