When Is a Relevance Threshold Statistically Resolvable? Minimax Limits for Effect Classification

Abstract

Statistical precision and scientific relevance operate on different scales. In regular problems, sampling uncertainty contracts at rate $n^{-1/2}$, whereas the magnitude below which an effect is scientifically negligible may be fixed or may vary with information. Let ${\Delta }_n$ denote a relevance threshold and $I_0$ Fisher information, and define ${\lambda }_n=\sqrt{n{I}_0}{\Delta }_n$. For separated negligible and meaningful parameter classes, we establish the minimax lower bound $\mathrm{liminf}{R}_n^{\ast }\ge 2\{1-\Phi (\epsilon \kappa )\}$ when ${\lambda }_n\to \kappa$. If ${\lambda }_n\to 0$, the experiments merge and consistent classification is impossible. If ${\lambda }_n\to \kappa \in (0,\infty )$, the problem converges to a Gaussian-shift decision problem. We characterize the exact minimax rule and risk in that limiting composite problem. As $\kappa \downarrow 0$, the optimal cutoff converges to one statistical-error unit and the optimal improvement over trivial risk is $O({\kappa }^2)$, while a simple relevance-boundary rule improves only at $O({\kappa }^3)$; as $\kappa$ grows, that rule becomes asymptotically minimax. If ${\lambda }_n\to \infty$, consistent classification is attainable under a uniform estimation-resolution condition, verified for Gaussian and Bernoulli models. For ${\Delta }_n=d{n}^{-\gamma }$, the critical rate is $\gamma =1/2$. We also show that moving asymmetric relevance regions depend on standardized distances to their two boundaries rather than total width alone. For heterogeneous true effects, we derive a significance-saturation limit and show that point-null significance can be most selective for scientifically relevant effects at intermediate information. The framework characterizes when scientific relevance is statistically resolvable through a common information scale.

arXiv comment30 pages, 4 figures