Partial information decomposition: redundancy as information bottleneck
arXiv:2405.07665 · doi:10.3390/e26070546
Abstract
The partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the "redundancy bottleneck" (RB). The RB formalizes a tradeoff between prediction and compression: it extracts information from the sources that best predict the target, without revealing which source provided the information. It can be understood as a generalization of "Blackwell redundancy", which we previously proposed as a principled measure of PID redundancy. The "RB curve" quantifies the prediction--compression tradeoff at multiple scales. This curve can also be quantified for individual sources, allowing subsets of redundant sources to be identified without combinatorial optimization. We provide an efficient iterative algorithm for computing the RB curve.
arXiv commentEntropy, 2024
References in corpus (17)
- Quantifying unique information
- Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
- Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
- Intersection Information based on Common Randomness
- A Novel Approach to the Partial Information Decomposition
- Understanding interdependency through complex information sharing
- Quantifying Redundant Information in Predicting a Target Random Variable
- Fast and exact search for the partition with minimal information loss
- On extractable shared information
- The Convex Information Bottleneck Lagrangian
- Caveats for information bottleneck in deterministic scenarios
- Coarse-graining and the Blackwell order
- Unique Informations and Deficiencies
- The Blackwell relation defines no lattice
- Partial Information Decomposition via Deficiency for Multivariate Gaussians
- The Variational Deficiency Bottleneck
- The Cardinality Bound on the Information Bottleneck Representations is Tight