Entropy Rate Estimation for Markov Chains with Large State Space

Abstract

Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on $S$ elements with independent samples, [Paninski2004] showed that the sample complexity is sublinear in $S$, and [Valiant--Valiant2011] showed that consistent estimation of Shannon entropy is possible if and only if the sample size $n$ far exceeds $\frac{S}{\log S}$. In this paper we consider the problem of estimating the entropy rate of a stationary reversible Markov chain with $S$ states from a sample path of $n$ observations. We show that: (1) As long as the Markov chain mixes not too slowly, i.e., the relaxation time is at most $O(\frac{S}{{\ln }^{3}S})$, consistent estimation is achievable when $n\gg \frac{S^2}{\log S}$. (2) As long as the Markov chain has some slight dependency, i.e., the relaxation time is at least $1+\Omega (\frac{{\ln }^{2}S}{\sqrt{S}})$, consistent estimation is impossible when $n\lesssim \frac{S^2}{\log S}$. Under both assumptions, the optimal estimation accuracy is shown to be $\Theta (\frac{S^2}{n\log S})$. In comparison, the empirical entropy rate requires at least $\Omega (S^2)$ samples to be consistent, even when the Markov chain is memoryless. In addition to synthetic experiments, we also apply the estimators that achieve the optimal sample complexity to estimate the entropy rate of the English language in the Penn Treebank and the Google One Billion Words corpora, which provides a natural benchmark for language modeling and relates it directly to the widely used perplexity measure.

arXiv commentPublished as a conference paper on NIPS 2018

References in corpus (8)

• Universal Estimation of Directed Information
• Does Dirichlet Prior Smoothing Solve the Shannon Entropy Estimation Problem?
• Maximum Likelihood Estimation of Functionals of Discrete Distributions
• The predictability of consumer visitation patterns
• Patterns of Individual Shopping Behavior
• Predictability of conversation partners
• Testing Symmetric Markov Chains from a Single Trajectory
• Estimating the Spectral Gap of a Reversible Markov Chain from a Short Trajectory