Adaptive Estimators Show Information Compression in Deep Neural Networks

Ivan Chelombiev, Conor Houghton, Cian O'Donnell

To improve how neural networks function it is crucial to understand their learning process. The information bottleneck theory of deep learning proposes that neural networks achieve…