Quantized Variational Inference

Abstract

We present Quantized Variational Inference, a new algorithm for Evidence Lower Bound maximization. We show how Optimal Voronoi Tesselation produces variance free gradients for ELBO optimization at the cost of introducing asymptotically decaying bias. Subsequently, we propose a Richardson extrapolation type method to improve the asymptotic bound. We show that using the Quantized Variational Inference framework leads to fast convergence for both score function and the reparametrized gradient estimator at a comparable computational cost. Finally, we propose several experiments to assess the performance of our method and its limitations.

References in corpus (11)

• Auto-Encoding Variational Bayes
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• MCMC using Hamiltonian dynamics
• A Conceptual Introduction to Hamiltonian Monte Carlo
• Variational Dropout and the Local Reparameterization Trick
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference
• Reducing Reparameterization Gradient Variance
• Overdispersed Black-Box Variational Inference
• New Weak Error bounds and expansions for Optimal Quantization
• Provable Gradient Variance Guarantees for Black-Box Variational Inference