Maximizing Multi-Information

Abstract

Stochastic interdependence of a probablility distribution on a product space is measured by its Kullback-Leibler distance from the exponential family of product distributions (called multi-information). Here we investigate low-dimensional exponential families that contain the maximizers of stochastic interdependence in their closure. Based on a detailed description of the structure of probablility distributions with globally maximal multi-information we obtain our main result: The exponential family of pure pair-interactions contains all global maximizers of the multi-information in its closure.

22 pages, 4 figures

Cited by in corpus (13)

• Expressive Power and Approximation Errors of Restricted Boltzmann Machines
• Complexity Measures from Interaction Structures
• Entropy Distance: New Quantum Phenomena
• Computing complexity measures for quantum states based on exponential families
• Quantum Convex Support
• Neighborliness of Marginal Polytopes
• Maximizing the divergence from a hierarchical model of quantum states
• Discrete Restricted Boltzmann Machines
• Mixture decompositions of exponential families using a decomposition of their sample spaces
• Hierarchical Models, Marginal Polytopes, and Linear Codes
• Analysing multiparticle quantum states
• Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units
• Maximizing the Bregman divergence from a Bregman family