Estimation of entropy-type integral functionals

Abstract

Entropy-type integral functionals of densities are widely used in mathematical statistics, information theory, and computer science. Examples include measures of closeness between distributions (e.g., density power divergence) and uncertainty characteristics for a random variable (e.g., Rényi entropy). In this paper, we study U-statistic estimators for a class of such functionals. The estimators are based on epsilon-close vector observations in the corresponding independent and identically distributed samples. We prove asymptotic properties of the estimators (consistency and asymptotic normality) under mild integrability and smoothness conditions for the densities. The results can be applied in diverse problems in mathematical statistics and computer science (e.g., distribution identification problems, approximate matching for random databases, two-sample problems).

References in corpus (1)

• Statistical Inference for Rényi Entropy Functionals

Cited by in corpus (4)

• Influence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations
• Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence
• Divergence Measures Estimation and Its Asymptotic Normality Theory Using Wavelets Empirical Processes
• Divergence measures estimation and its asymptotic normality theory in the discrete case