Information theory and learning: a physical approach

Ilya Nemenman

We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual inform…