Variational Inference with Tail-adaptive f-Divergence

Abstract

Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importance sampling, which could have extremely large or infinite variances due to heavy tails of importance weights. In this paper, we propose a new class of tail-adaptive f-divergences that adaptively change the convex function f with the tail of the importance weights, in a way that theoretically guarantees finite moments, while simultaneously achieving mass-covering properties. We test our methods on Bayesian neural networks, as well as deep reinforcement learning in which our method is applied to improve a recent soft actor-critic (SAC) algorithm. Our results show that our approach yields significant advantages compared with existing methods based on classical KL and α-divergences.

arXiv commentNeurIPS 2018

Cited by in corpus (12)

• Generalized Variational Inference: Three arguments for deriving new Posteriors
• An introduction to variational inference in Geophysical inverse problems
• f-Divergence Variational Inference
• Infinite-dimensional gradient-based descent for alpha-divergence minimisation
• Challenges and Opportunities in High-dimensional Variational Inference
• On the Difficulty of Unbiased Alpha Divergence Minimization
• Generalized Bayesian Filtering via Sequential Monte Carlo
• On Proximal Policy Optimization's Heavy-tailed Gradients
• Mixture weights optimisation for Alpha-Divergence Variational Inference
• All in the Exponential Family: Bregman Duality in Thermodynamic Variational Inference
• Doubly Adaptive Importance Sampling
• Scalable Approximate Inference and Some Applications