Chernoff's bound forms

Abstract

Chernoff's bound binds a tail probability (ie. $Pr(X\ge a)$, where $a\ge EX$). Assuming that the distribution of $X$ is $Q$, the logarithm of the bound is known to be equal to the value of relative entropy (or minus Kullback-Leibler distance) for $I$-projection $\hat{P}$ of $Q$ on a set $\mathcal{H}\triangleq \{P:E_{P}X=a\}$. Here, Chernoff's bound is related to Maximum Likelihood on exponential form and consequently implications for the notion of complementarity are discussed. Moreover, a novel form of the bound is proposed, which expresses the value of the Chernoff's bound directly in terms of the $I$-projection (or generalized $I$-projection).

M. Grendar, Jr. and M. Grendar, ``Chernoff's bound forms,'' in Bayesian inference and Maximum Entropy methods in Science and Engineering, edited by Ch. Williams, AIP Conference Proceedings 659, Melville, New York, 2003, pp. 67-72

References in corpus (2)

• What is the question that MaxEnt answers? A probabilistic interpretation
• Minimax Entropy and Maximum Likelihood. Complementarity of tasks, identity of solutions