An Exploration of the Role of Principal Inertia Components in Information Theory

Abstract

The principal inertia components of the joint distribution of two random variables $X$ and $Y$ are inherently connected to how an observation of $Y$ is statistically related to a hidden variable $X$. In this paper, we explore this connection within an information theoretic framework. We show that, under certain symmetry conditions, the principal inertia components play an important role in estimating one-bit functions of $X$, namely $f(X)$, given an observation of $Y$. In particular, the principal inertia components bear an interpretation as filter coefficients in the linear transformation of $p_{f(X)|X}$ into $p_{f(X)|Y}$. This interpretation naturally leads to the conjecture that the mutual information between $f(X)$ and $Y$ is maximized when all the principal inertia components have equal value. We also study the role of the principal inertia components in the Markov chain $B\to X\to Y\to \hat{B}$, where $B$ and $\hat{B}$ are binary random variables. We illustrate our results for the setting where $X$ and $Y$ are binary strings and $Y$ is the result of sending $X$ through an additive noise binary channel.

Submitted to the 2014 IEEE Information Theory Workshop (ITW)

References in corpus (1)

• An Exploration of the Role of Principal Inertia Components in Information Theory

Cited by in corpus (6)

• An Exploration of the Role of Principal Inertia Components in Information Theory
• Maximum Entropy Functions: Approximate Gacs-Korner for Distributed Compression
• Hiding Symbols and Functions: New Metrics and Constructions for Information-Theoretic Security
• On the Courtade-Kumar conjecture for certain classes of Boolean functions
• The maximum mutual information between the output of a discrete symmetric channel and several classes of Boolean functions of its input
• The maximum mutual information between the output of a binary symmetric channel and a Boolean function of its input