Numerical evaluation of Gaussian mixture entropy

Abstract

We develop an approximation method for the differential entropy $h(\mathbf{X})$ of a $q$-component Gaussian mixture in ${\mathbb{R}}^n$. We provide two examples of approximations using our method denoted by ${\bar{h}}_{C,m}^{\mathrm{Taylor}}(\mathbf{X})$ and ${\bar{h}}_{C,m}^{\mathrm{Polyfit}}(\mathbf{X})$. We show that ${\bar{h}}_{C,m}^{\mathrm{Taylor}}(\mathbf{X})$ provides an easy to compute lower bound to $h(\mathbf{X})$, while ${\bar{h}}_{C,m}^{\mathrm{Polyfit}}(\mathbf{X})$ provides an accurate and efficient approximation to $h(\mathbf{X})$. ${\bar{h}}_{C,m}^{\mathrm{Polyfit}}(\mathbf{X})$ is more accurate than known bounds, and conjectured to be much more resilient than the approximation of [5] in high dimensions.