Directional replicability: when can the factor of two be omitted

Abstract

Directional replicability addresses the question of whether an effect studied across $n$ independent studies is present with the same direction in at least $r$ of them, for $r\ge 2$. When the expected direction of the effect is not specified in advance, the state of the art recommends assessing replicability separately by combining one-sided $p$-values for both directions (left and right), and then doubling the smaller of the two resulting combined $p$-values to account for multiple testing. In this work, we show that this multiplicative correction is not always necessary, and give conditions under which it can be safely omitted.