Finding the Maximizers of the Information Divergence from an Exponential Family

Abstract

This paper investigates maximizers of the information divergence from an exponential family $E$. It is shown that the $rI$-projection of a maximizer $P$ to $E$ is a convex combination of $P$ and a probability measure $P_{-}$ with disjoint support and the same value of the sufficient statistics $A$. This observation can be used to transform the original problem of maximizing $D(\cdot ||E)$ over the set of all probability measures into the maximization of a function $\Dbar$ over a convex subset of $\ker A$. The global maximizers of both problems correspond to each other. Furthermore, finding all local maximizers of $\Dbar$ yields all local maximizers of $D(\cdot ||E)$. This paper also proposes two algorithms to find the maximizers of $\Dbar$ and applies them to two examples, where the maximizers of $D(\cdot ||E)$ were not known before.

25 pages

References in corpus (2)

• Support Sets in Exponential Families and Oriented Matroid Theory
• Decompositions of Binomial Ideals

Cited by in corpus (6)

• On the Entropy of Couplings
• Entropy Distance: New Quantum Phenomena
• Maximizing the divergence from a hierarchical model of quantum states
• A new signature of quantum phase transitions from the numerical range
• A variation principle for ground spaces
• Maximal Information Divergence from Statistical Models defined by Neural Networks