On Dependence Measures Based on $f$-Divergence and $\Phi$-Entropy

Abstract

We study dependence measures induced by $\Phi$-divergence (also known as $f$-divergence in the literature) for finite random variables. We define conditional $\Phi$-mutual information through an exact chain rule and develop its structural properties and variational representations. But in general it lacks conditional symmetry and deterministic self-reduction. To tackle this, we introduce max-$\Phi$-mutual information $\wIPhi$, a variational envelope obtained by optimizing over $\IPhi$ quantities containing auxiliary random variables, which restores both properties and satisfies a subadditive chain rule. Both quantities reduce to Shannon's mutual information when $\Phi (t)=t\log t$. For smooth convex generators with $\Phi (0)<\infty$ and concave $t\mapsto t^2{\Phi }^{\prime \prime }(t)$, we establish an optimizer characterization and obtain a closed-form expression for the unconditional quantity in terms of $\Phi$-self-information. For the same class, we show that its private-randomness-assisted version is either an affine function of Shannon's mutual information, with coefficients depending only on $\Phi$, or identically infinite. This dichotomy is determined by the asymptotic growth of $\Phi$ and clarifies the sensitivity of max-$\Phi$-mutual information to local randomness. We further derive inner bounds on $\Phi$-ribbons from prescribed independence structures and prove that these bounds are optimal when $\Phi (t)=t\log t$. We also establish a multipartite $\Phi$-generalization of the Zhang--Yeung inequality and derive corresponding ribbon regions. Finally, we give sufficient conditions for reducing the computation of $\Phi$-strong data processing constants for Z-channel sources to a boundary optimization.