Characterization of Conditional Independence and Weak Realizations of Multivariate Gaussian Random Variables: Applications to Networks

Abstract

The Gray and Wyner lossy source coding for a simple network for sources that generate a tuple of jointly Gaussian random variables (RVs) $X_1:\Omega \to {\mathbb{R}}^{p_1}$ and $X_2:\Omega \to {\mathbb{R}}^{p_2}$, with respect to square-error distortion at the two decoders is re-examined using (1) Hotelling's geometric approach of Gaussian RVs-the canonical variable form, and (2) van Putten's and van Schuppen's parametrization of joint distributions ${\mathbf{P}}_{X_1,X_2,W}$ by Gaussian RVs $W:\Omega \to {\mathbb{R}}^n$ which make $(X_1,X_2)$ conditionally independent, and the weak stochastic realization of $(X_1,X_2)$. Item (2) is used to parametrize the lossy rate region of the Gray and Wyner source coding problem for joint decoding with mean-square error distortions $\mathbf{E}\{||X_i-{\hat{X}}_i|{|}_{{\mathbb{R}}^{p_i}}^2\}\le {\Delta }_i\in [0,\infty ],i=1,2$, by the covariance matrix of RV $W$. From this then follows Wyner's common information $C_W(X_1,X_2)$ (information definition) is achieved by $W$ with identity covariance matrix, while a formula for Wyner's lossy common information (operational definition) is derived, given by $C_{WL}(X_1,X_2)=C_W(X_1,X_2)=\frac{1}{2}{\sum }_{j=1}^n\ln \left(\frac{1+d_j}{1-d_j}\right),$ for the distortion region $0\le {\Delta }_1\le {\sum }_{j=1}^n(1-d_j)$, $0\le {\Delta }_2\le {\sum }_{j=1}^n(1-d_j)$, and where $1>d_1\ge d_2\ge \dots \ge d_n>0$ in $(0,1)$ are {\em the canonical correlation coefficients} computed from the canonical variable form of the tuple $(X_1,X_2)$. The methods are of fundamental importance to other problems of multi-user communication, where conditional independence is imposed as a constraint.

6 pages