Capacity of Random Channels with Large Alphabets

Abstract

We consider discrete memoryless channels with input alphabet size $n$ and output alphabet size $m$, where $m=$ceil$({\hat{I}}^{3}n)$ for some constant ${\hat{I}}^3>0$. The channel transition matrix consists of entries that, before being normalised, are independent and identically distributed nonnegative random variables $V$ and such that $E[(V\log V{)}^2]<\infty$. We prove that in the limit as $n\to \infty$ the capacity of such a channel converges to $Ent(V)/E[V]$ almost surely and in $L^2$, where $Ent(V):=E[V\log V]-E[V]\log E[V]$ denotes the entropy of $V$. We further show that, under slightly different model assumptions, the capacity of these random channels converges to this asymptotic value exponentially in $n$. Finally, we present an application in the context of Bayesian optimal experiment design.

20 pages, 2 figures, revised version