Universal Inference

Abstract

We propose a general method for constructing hypothesis tests and confidence sets that have finite sample guarantees without regularity conditions. We refer to such procedures as "universal." The method is very simple and is based on a modified version of the usual likelihood ratio statistic, that we call "the split likelihood ratio test" (split LRT). The method is especially appealing for irregular statistical models. Canonical examples include mixture models and models that arise in shape-constrained inference. Constructing tests and confidence sets for such models is notoriously difficult. Typical inference methods, like the likelihood ratio test, are not useful in these cases because they have intractable limiting distributions. In contrast, the method we suggest works for any parametric model and also for some nonparametric models. The split LRT can also be used with profile likelihoods to deal with nuisance parameters, and it can also be run sequentially to yield anytime-valid $p$-values and confidence sequences.

arXiv commentTo appear in the Proceedings of the National Academy of Sciences

References in corpus (13)

• Hypothesis test for normal mixture models: The EM approach
• Tightness of the maximum likelihood semidefinite relaxation for angular synchronization
• Likelihood ratio tests and singularities
• Time-uniform Chernoff bounds via nonnegative supermartingales
• Test Martingales, Bayes Factors and $p$-Values
• Global analysis of Expectation Maximization for mixtures of two Gaussians
• Nonanticipating estimation applied to sequential analysis and changepoint detection
• Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences
• Sequential estimation of quantiles with applications to A/B-testing and best-arm identification
• Assigning a value to a power likelihood in a general Bayesian model
• On Testing Marginal versus Conditional Independence
• Gaussian Mixture Clustering Using Relative Tests of Fit
• A Polynomial Time Algorithm for Log-Concave Maximum Likelihood via Locally Exponential Families

Cited by in corpus (13)

• E-values: Calibration, combination, and applications
• Significance tests of feature relevance for a black-box learner
• Total positivity in exponential families with application to binary variables
• A Note on Likelihood Ratio Tests for Models with Latent Variables
• On the choice of the splitting ratio for the split likelihood ratio test
• Confidence sequences for sampling without replacement
• Post-selection inference for e-value based confidence intervals
• On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning
• Uncertainty quantification using martingales for misspecified Gaussian processes
• Confidence in Causal Discovery with Linear Causal Models
• Finite sample inference for generic autoregressive models
• Testing exchangeability: fork-convexity, supermartingales, and e-processes
• Non-Asymptotic Bounds for the ${\ell }_{\infty }$ Estimator in Linear Regression with Uniform Noise