Entropy and mutual information in models of deep neural networks

Marylou Gabrié, Andre Manoel, Clément Luneau +4

We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutu…