On the Information Bottleneck for Stable Variables

Abstract

We consider a generalization of the Gaussian information bottleneck problem to jointly stable variables $X$ and $Y$. Specifically, for $X=Y+A$, where $A$ is stable and independent of $Y$, we find optimal stable linear encoders of the form $T=kX+N$, with $N$ stable and independent of $X$. We characterize the linear stable information bottleneck in closed-form and its critical value. Furthermore, we show that, except for the Gaussian case, such stochastic linear encoders are strictly suboptimal, while becoming asymptotically optimal at large compression rates. Our linear solution recovers the Gaussian information bottleneck for jointly Gaussian variables as a special case.

arXiv comment13 pages, 3 figures