Variational inference based on a subclass of closed skew normals

Abstract

Gaussian distributions are widely used in Bayesian variational inference to approximate intractable posterior densities, but the ability to accommodate skewness can improve approximation accuracy significantly, when data or prior information is scarce. We study the properties of a subclass of closed skew normals constructed using affine transformation of independent standardized univariate skew normals as the variational density, and illustrate how it provides increased flexibility and accuracy in approximating the joint posterior in various applications, by overcoming limitations in existing skew normal variational approximations. The evidence lower bound is optimized using stochastic gradient ascent, where analytic natural gradient updates are derived. We also demonstrate how problems in maximum likelihood estimation of skew normal parameters occur similarly in stochastic variational inference, and can be resolved using the centered parametrization. Supplemental materials are available online.

keywords: Closed skew normal; Gaussian variational approximation; natural gradient; centered parametrization; LU decomposition

References in corpus (16)

• Variational Inference: A Review for Statisticians
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Statistical applications of the multivariate skew-normal distribution
• Inferring Parameters and Structure of Latent Variable Models by Variational Bayes
• Normalizing Flows for Probabilistic Modeling and Inference
• Automatic Differentiation Variational Inference
• Frequentist Consistency of Variational Bayes
• Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
• Variational Bayes for high-dimensional linear regression with sparse priors
• Gaussian variational approximation with sparse precision matrices
• normflows: A PyTorch Package for Normalizing Flows
• Conditionally conjugate mean-field variational Bayes for logistic models
• Skew-symmetric distributions and Fisher information: The double sin of the skew-normal
• Variational Inference with Continuously-Indexed Normalizing Flows
• Variance reduction properties of the reparameterization trick
• Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation

Cited by in corpus (1)

• Variational inference for hierarchical models with conditional scale and skewness corrections