Closed-Form Information-Theoretic Roughness Measures for Mixture Densities

Abstract

We calculate the smoothest mixture density under a variety of prescribed specifications. This includes constraints on certain moments, specifications on density values and/or its derivatives, and prescribed probability masses in certain regions. As a roughness measure, we use Fisher Information (FI) in the space of mixtures $\mathcal{M}$. For mixtures, FI cannot be calculated in closed form. We define the space $\mathcal{R}$ of root mixtures (RMs) living on the Hilbert sphere. A transformation of FI to $\mathcal{R}$ admits a closed-form solution and yields the desired result in $\mathcal{M}$. This naturally leads to a tandem processing with two density representations maintained simultaneously in $\mathcal{R}$ and $\mathcal{M}$. FI is calculated in RM space $\mathcal{R}$ while the constraints are evaluated in mixture space $\mathcal{M}$.