Nearest neighbor density functional estimation from inverse Laplace transform

Abstract

A new approach to $L_2$-consistent estimation of a general density functional using $k$-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$-nearest neighbor ball to a Gamma distribution in the large-sample limit, and naturally involves the inverse Laplace transform of a scaled version of the function $f.$ Some instantiations of the proposed estimator recover existing $k$-nearest neighbor based estimators of Shannon and Rényi entropies and Kullback--Leibler and Rényi divergences, and discover new consistent estimators for many other functionals such as logarithmic entropies and divergences. The $L_2$-consistency of the proposed estimator is established for a broad class of densities for general functionals, and the convergence rate in mean squared error is established as a function of the sample size for smooth, bounded densities.

arXiv comment43 pages, 4 figures. IEEE Transactions on Information Theory (to appear)

References in corpus (10)

• Estimating Mutual Information
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Fast learning rates for plug-in classifiers
• A class of Rényi information estimators for multidimensional densities
• Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs
• Ensemble estimation of multivariate f-divergence
• The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
• Direct Estimation of Information Divergence Using Nearest Neighbor Ratios
• Ensemble Estimation of Information Divergence
• Independent Component Analysis by Entropy Maximization with Kernels

Cited by in corpus (2)

• Towards Robust Data-Driven Underwater Acoustic Localization: A Deep CNN Solution with Performance Guarantees for Model Mismatch
• Ensemble Estimation of Generalized Mutual Information with Applications to Genomics