Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning

Andreas Kirsch, Clare Lyle, Yarin Gal

The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models…