Usable Information and Evolution of Optimal Representations During Training

Michael Kleinman, Alessandro Achille, Daksh Idnani +1

We introduce a notion of usable information contained in the representation learned by a deep network, and use it to study how optimal representations for the task emerge during tr…