Variational Predictive Information Bottleneck

Abstract

In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional. Below we derive a generalized form of this functional as a variational lower bound of a predictive information bottleneck objective. This generalized functional encompasses most modern inference procedures and suggests novel ones.

References in corpus (7)

• The information bottleneck method
• The formal definition of reference priors
• Deep Variational Information Bottleneck
• Maximizing the information learned from finite data selects a simple model
• Generalized Variational Inference: Three arguments for deriving new Posteriors
• Learners that Use Little Information
• Variational Inference based on Robust Divergences

Cited by in corpus (2)

• General Information Bottleneck Objectives and their Applications to Machine Learning
• Machine Learning using the Variational Predictive Information Bottleneck with a Validation Set