Compression Through Time: An Interplay of Separability, Stationarity and Capacity

Nikolaus Kopp, Franz Pernkopf, Bernhard Geiger

Information-theoretic and geometric compression have been widely studied as possible explanations for generalization. Yet their temporal dynamics in recurrent networks remain large…