The Capacity of Finite-State Channels in the High-Noise Regime

Abstract

This paper considers the derivative of the entropy rate of a hidden Markov process with respect to the observation probabilities. The main result is a compact formula for the derivative that can be evaluated easily using Monte Carlo methods. It is applied to the problem of computing the capacity of a finite-state channel (FSC) and, in the high-noise regime, the formula has a simple closed-form expression that enables series expansion of the capacity of a FSC. This expansion is evaluated for a binary-symmetric channel under a (0,1) run-length limited constraint and an intersymbol-interference channel with Gaussian noise.

arXiv commentExtended version of research presented at: "Entropy of Hidden Markov Processes and Connections to Dynamical Systems", Banff International Reseach Station (BIRS), October 5, 2007