Average entropy of Gaussian mixtures

Abstract

We calculate the average differential entropy of a $q$-component Gaussian mixture in ${\mathbb{R}}^n$. For simplicity, all components have covariance matrix $\ddot{I}{ƒ}^{2}1$, while the means $\{{\mathbf{W}}_i{\}}_{i=1}^q$ are i.i.d. Gaussian vectors with zero mean and covariance $s^{2}1$. We obtain a series expansion in $\hat{I}¼=s^2/\ddot{I}{ƒ}^2$ for the average differential entropy up to order $\mathcal{O}(\hat{I}{¼}^2)$, and we provide a recipe to calculate higher order terms. Our result provides an analytic approximation with a quantifiable order of magnitude for the error, which is not achieved in previous literature.

24 pages