2.5k citations · 4.7k across the 86 of their papers we have counts for
8 papers · 1 filter
Inferring Local Structure from Pairwise Correlations
Mahajabin Rahman, Ilya Nemenman
To construct models of large, multivariate complex systems, such as those in biology, one needs to constrain which variables are allowed to interact. This can be viewed as detectin…
Low probability states, data statistics, and entropy estimation
Damián G. Hernández, Ahmed Roman, Ilya Nemenman
A fundamental problem in analysis of complex systems is getting a reliable estimate of entropy of their probability distributions over the state space. This is difficult because un…
Entropy and information in neural spike trains: Progress on the sampling problem
Ilya Nemenman, William Bialek, Rob de Ruyter van Steveninck
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We appl…
Inference of entropies of discrete random variables with unknown cardinalities
Ilya Nemenman
We examine the recently introduced NSB estimator of entropies of severely undersampled discrete variables and devise a procedure for calculating the involved integrals. We discover…
Entropy and inference, revisited
Ilya Nemenman, Fariel Shafee, William Bialek
We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrou…
Complexity Through Nonextensivity
William Bialek, Ilya Nemenman, Naftali Tishby
The problem of defining and studying complexity of a time series has interested people for years. In the context of dynamical systems, Grassberger has suggested that a slow approac…
Information theory and learning: a physical approach
Ilya Nemenman
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual inform…
Predictability, complexity and learning
William Bialek, Ilya Nemenman, Naftali Tishby
We define {\em predictive information} as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are fo…