2.5k citations · 4.7k across the 86 of their papers we have counts for
Showing 2000Show all
3 papers · 1 filter
physics.data-an2000★ 14 cited
Information theory and learning: a physical approach
Ilya Nemenman
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual inform…
cond-mat2000★ 33 cited
Occam factors and model-independent Bayesian learning of continuous distributions
Ilya Nemenman, William Bialek
Learning of a smooth but nonparametric probability density can be regularized using methods of Quantum Field Theory. We implement a field theoretic prior numerically, test its effi…
physics.data-an2000★ 2 cited
Predictability, complexity and learning
William Bialek, Ilya Nemenman, Naftali Tishby
We define {\em predictive information} as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are fo…