Exploring Consequences of Simulation Design for Apparent Performance of Statistical Methods. 1: Results from simulations with constant sample sizes
arXiv:2006.16638
Abstract
Contemporary statistical publications rely on simulation to evaluate performance of new methods and compare them with established methods. In the context of meta-analysis of log-odds-ratios, we investigate how the ways in which simulations are implemented affect such conclusions. Choices of distributions for sample sizes and/or control probabilities considerably affect conclusions about statistical methods. Here we report on the results for constant sample sizes. Our two subsequent publications will cover normally and uniformly distributed sample sizes.
8 pages and full simulation results, comprising 400 figures, each presenting 12 combinations of sample sizes and numbers of studies