SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching

Hippolyte Labarrière, Cesare Molinari, Silvia Villa +1

Black-Box Variational Inference (BBVI) typically relies on Stochastic Gradient Descent (SGD) to optimize the Evidence Lower Bound (ELBO). However, the stochastic gradients in BBVI…