On shared and multiple information

Abstract

We address three outstanding problems in information theory. Problem one is the definition of a non-negative decomposition of the information conveyed by two or more sources about a target variable into the specific contribution of each possible combination of the sources. Problem two is the definition of a measure of information shared by several sources about the target variable. Problem three is the definition of a measure of multiple mutual information, that is, the extension of mutual information to more than two variables.

Added proof of additivity property

References in corpus (3)

• Nonnegative Decomposition of Multivariate Information
• Multivariate Dependence Beyond Shannon Information
• Redundancy and synergy in dual decompositions of mutual information gain and information loss