Entropy and information in neural spike trains: Progress on the sampling problem

Abstract

The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to synthetic data inspired by experiments, and to real experimental spike trains. The estimator performs admirably even very deep in the undersampled regime, where other techniques fail. This opens new possibilities for the information theoretic analysis of experiments, and may be of general interest as an example of learning from limited data.

7 pages, 4 figures; referee suggested changes, accepted version

References in corpus (3)

• Inference of entropies of discrete random variables with unknown cardinalities
• Information theory and learning: a physical approach
• Timing and Counting Precision in the Blowfly Visual System

Cited by in corpus (60)

• Nonlinear multivariate analysis of Neurophysiological Signals
• Equitability, mutual information, and the maximal information coefficient
• Mutual Information Rate-Based Networks in Financial Markets
• Motor control by precisely timed spike patterns
• Zipf's law and criticality in multivariate data without fine-tuning
• Maximum Likelihood Estimation of Functionals of Discrete Distributions
• Does Dirichlet Prior Smoothing Solve the Shannon Entropy Estimation Problem?
• Entropy estimates of small data sets
• Estimating the entropy of binary time series: Methodology, some theory and a simulation study
• On integral probability metrics, ϕ-divergences and binary classification
• Statistical mechanics of the US Supreme Court
• Inferring interaction partners from protein sequences using mutual information
• Bootstrap Methods for the Empirical Study of Decision-Making and Information Flows in Social Systems
• Inference of entropies of discrete random variables with unknown cardinalities
• Breakdown of local information processing may underlie isoflurane anesthesia effects
• Identification of functional information subgraphs in complex networks
• Energy efficiency of information transmission by electrically coupled neurons
• Burst firing is a neural code in an insect auditory system
• Topological Information Data Analysis
• Multivariate dependence and genetic networks inference
• Information flow through a model of the C. elegans klinotaxis circuit
• Physical constraints in intracellular signaling: the cost of sending a bit
• Information estimation using nonparametric copulas
• Understanding the Limitations of Variational Mutual Information Estimators
• Information theory, multivariate dependence, and genetic network inference
• Certification of high-dimensional entanglement and Einstein-Podolsky-Rosen steering with cold atomic quantum memory
• Information-theoretic approach to lead-lag effect on financial markets
• Rényi entropy, abundance distribution and the equivalence of ensembles
• Dynamics and Processing in Finite Self-Similar Networks
• Estimation of the Rate-Distortion Function
• Entropy evaluation sheds light on ecosystem complexity
• Estimating the Mutual Information between two Discrete, Asymmetric Variables with Limited Samples
• Wasserstein Dependency Measure for Representation Learning
• The behaviour of information flow near criticality
• Posterior Representations for Bayesian Context Trees: Sampling, Estimation and Convergence
• Entropy estimators for Markovian sequences: A comparative analysis
• Estimating Functions of Distributions Defined over Spaces of Unknown Size
• Polynomial methods in statistical inference: theory and practice
• Time and category information in pattern-based codes
• Information transfer with small-amplitude signals
• Partial Mutual Information Analysis of Financial Networks
• A Kernel-Based Calculation of Information on a Metric Space
• The information transmitted by spike patterns in single neurons
• Minimum and maximum entropy distributions for binary systems with known means and pairwise correlations
• Approximations of Shannon Mutual Information for Discrete Variables with Applications to Neural Population Coding
• A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition
• Clarifying the conceptual dimensions of representation in neuroscience
• Pareto-optimal clustering with the primal deterministic information bottleneck
• Estimating Information-Theoretic Quantities
• On the Estimation of Information Measures of Continuous Distributions
• Low probability states, data statistics, and entropy estimation
• Bayesian nonparametric estimation of Tsallis diversity indices under Gnedin-Pitman priors
• Exploring language relations through syntactic distances and geographic proximity
• Neural coding of a natural stimulus ensemble: Uncovering information at sub-millisecond resolution
• Statistical Inference in the Differential Privacy Model
• Inferring a property of a large system from a small number of samples
• Information-theoretic analysis of temporal dependence in discrete stochastic processes: Application to precipitation predictability
• Measuring Grammatical Diversity from Small Corpora: Derivational Entropy Rates, Mean Length of Utterances, and Annotation Invariance
• Millisecond-scale motor encoding in a cortical vocal area
• Entropy, mutual information, and systematic measures of structured spiking neural networks