Information Dropout: Learning Optimal Representations Through Noisy Computation

Alessandro Achille, Stefano Soatto

The cross-entropy loss commonly used in deep learning is closely related to the defining properties of optimal representations, but does not enforce some of the key properties. We…