Universal Bounds for Self-Normalized Kullback-Leibler Estimation

Buu Phan, Lester Mackey, David Alvarez-Melis

This work studies an entropy-based estimator of Kullback-Leibler (KL) divergence constructed from normalized importance weights. We prove a universal finite-sample bound showing th…