A scaling law from discrete to continuous solutions of channel capacity problems in the low-noise limit
arXiv:1710.09351 · doi:10.1007/s10955-019-02296-2
Abstract
An analog communication channel typically achieves its full capacity when the distribution of inputs is discrete, composed of just K symbols, such as voltage levels or wavelengths. As the effective noise level goes to zero, for example by sending the same message multiple times, it is known that optimal codes become continuous. Here we derive a scaling law for the optimal number of symbols in this limit, finding a novel rational scaling exponent. The number of symbols in the optimal code grows as , where the channel capacity I increases with decreasing noise. The same scaling applies to other problems equivalent to maximizing channel capacity over a continuous distribution.
arXiv comment14 pages, 5 figures. v2: Journal version with extended introduction (and extended title, was "An information scaling law: ζ= 3/4")
References in corpus (6)
- Universally Sloppy Parameter Sensitivities in Systems Biology
- Parameter Space Compression Underlies Emergent Theories and Predictive Models
- How a well-adapted immune system is organized
- Maximizing the information learned from finite data selects a simple model
- On the correspondence between thermodynamics and inference
- Iterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk