Finite-sample concentration of the empirical relative entropy around its mean

Rohit Agrawal

In this note, we show that the relative entropy of an empirical distribution of $n$ samples drawn from a set of size $k$ with respect to the true underlying distribution is exponen…