Choosing alpha post hoc: the danger of multiple standard significance thresholds

Jesse Hemerik, Nick W Koning

A fundamental assumption of classical hypothesis testing is that the significance threshold $\alpha$ is chosen independently from the data. The validity of confidence intervals likewise…