Exact testing with random permutations

Abstract

When permutation methods are used in practice, often a limited number of random permutations are used to decrease the computational burden. However, most theoretical literature assumes that the whole permutation group is used, and methods based on random permutations tend to be seen as approximate. There exists a very limited amount of literature on exact testing with random permutations and only recently a thorough proof of exactness was given. In this paper we provide an alternative proof, viewing the test as a "conditional Monte Carlo test" as it has been called in the literature. We also provide extensions of the result. Importantly, our results can be used to prove properties of various multiple testing procedures based on random permutations.

References in corpus (3)

• In silico prediction of protein-protein interactions in human macrophages
• Permutation p-values should never be zero: calculating exact p-values when permutations are randomly drawn
• The sequential rejection principle of familywise error control

Cited by in corpus (14)

• Permutation-based simultaneous confidence bounds for the false discovery proportion
• Dwarf galaxies imply dark matter is heavier than $2.2\times 10^{-21}\mathbf{eV}$
• Statistical quantification of confounding bias in predictive modelling
• Comparing three groups
• Robust testing in generalized linear models by sign-flipping score contributions
• Another look at the Lady Tasting Tea and differences between permutation tests and randomization tests
• Post-selection Inference in Multiverse Analysis (PIMA): an inferential framework based on the sign flipping score test
• Confidence and discoveries with e-values
• Better-Than-Chance Classification for Signal Detection
• Permutation-Based True Discovery Guarantee by Sum Tests
• Functional Response Designs via the Analytic Permutation Test
• An approximate randomization test for high-dimensional two-sample Behrens-Fisher problem under arbitrary covariances
• CRP-Tree: A phylogenetic association test for binary traits
• On the term "randomization test"