Testing composite hypotheses, Hermite polynomials and optimal estimation of a nonsmooth functional

Abstract

A general lower bound is developed for the minimax risk when estimating an arbitrary functional. The bound is based on testing two composite hypotheses and is shown to be effective in estimating the nonsmooth functional $\frac{1}{n}\sum |{\theta }_i|$ from an observation $Y\sim N(\theta ,I_n)$. This problem exhibits some features that are significantly different from those that occur in estimating conventional smooth functionals. This is a setting where standard techniques fail to yield sharp results. A sharp minimax lower bound is established by applying the general lower bound technique based on testing two composite hypotheses. A key step is the construction of two special priors and bounding the chi-square distance between two normal mixtures. An estimator is constructed using approximation theory and Hermite polynomials and is shown to be asymptotically sharp minimax when the means are bounded by a given value $M$. It is shown that the minimax risk equals ${\beta }_{\ast }^2{M}^2(\frac{\log \log n}{\log n}{)}^2$ asymptotically, where ${\beta }_{\ast }$ is the Bernstein constant. The general techniques and results developed in the present paper can also be used to solve other related problems.

Published in at http://dx.doi.org/10.1214/10-AOS849 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (3)

• Effect of mean on variance function estimation in nonparametric regression
• Nonquadratic estimators of a quadratic functional
• Minimax estimation of linear functionals over nonconvex parameter spaces

Cited by in corpus (29)

• Estimation under group actions: recovering orbits from invariants
• Minimax Estimation of the $L_1$ Distance
• Chebyshev polynomials, moment matching, and optimal estimation of the unseen
• On Estimation of $L_r$-Norms in Gaussian White Noise Models
• Minimax rates of entropy estimation on large alphabets via best polynomial approximation
• Minimax Rate-Optimal Estimation of Divergences between Discrete Distributions
• Optimal estimation of Gaussian mixtures via denoised method of moments
• Beyond Maximum Likelihood: from Theory to Practice
• Polynomial methods in statistical inference: theory and practice
• Estimating Certain Integral Probability Metric (IPM) is as Hard as Estimating under the IPM
• Dualizing Le Cam's method for functional estimation, with applications to estimating the unseens
• Estimation of KL Divergence: Optimal Minimax Rate
• Minimax Optimal Estimators for Additive Scalar Functionals of Discrete Distributions
• Minimax Rates of Estimating Approximate Differential Privacy
• Estimating Learnability in the Sublinear Data Regime
• Counting Motifs with Graph Sampling
• Tight Lower Bound for Linear Sketches of Moments
• On the High Accuracy Limitation of Adaptive Property Estimation
• Treatment Effects Estimation by Uniform Transformer
• Lepski's Method and Adaptive Estimation of Nonlinear Integral Functionals of Density
• On the Competitive Analysis and High Accuracy Optimality of Profile Maximum Likelihood
• Locally differentially private estimation of nonlinear functionals of discrete distributions
• Estimating the number of unseen species: A bird in the hand is worth $\log n$ in the bush
• Optimal Estimation of Simultaneous Signals Using Absolute Inner Product with Applications to Integrative Genomics
• Minimax Euclidean Separation Rates for Testing Convex Hypotheses in ${\mathbb{R}}^d$
• Optimal Estimation of Schatten Norms of a rectangular Matrix
• Learning Minimax Estimators via Online Learning
• Minimax optimal estimators for general additive functional estimation
• Minimax $L_2$-Separation Rate in Testing the Sobolev-Type Regularity of a function