A Plea for Neutral Comparison Studies in Computational Sciences

Abstract

In a context where most published articles are devoted to the development of "new methods", comparison studies are generally appreciated by readers but surprisingly given poor consideration by many scientific journals. In connection with recent articles on over-optimism and epistemology published in Bioinformatics, this letter stresses the importance of neutral comparison studies for the objective evaluation of existing methods and the establishment of standards by drawing parallels with clinical research.

References in corpus (1)

• Machine Learning that Matters

Cited by in corpus (13)

• A review on longitudinal data analysis with random forest in precision medicine
• Deep Learning for Survival Analysis: A Review
• Large-scale benchmark study of survival prediction methods using multi-omics data
• Phases of methodological research in biostatistics - building the evidence base for new methods
• Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method
• Benchmarking in cluster analysis: A white paper
• Explaining the optimistic performance evaluation of newly proposed methods: a cross-design validation experiment
• Simulation study to evaluate when Plasmode simulation is superior to parametric simulation in estimating the mean squared error of the least squares estimator in linear regression
• Over-optimism in benchmark studies and the multiplicity of design and analysis options when interpreting their results
• On "Confirmatory" Methodological Research in Statistics and Related Fields
• Unsupervised Functional Data Analysis via Nonlinear Dimension Reduction
• Guidelines for benchmarking of optimization approaches for fitting mathematical models
• Outcomes truncated by death in RCTs: a simulation study on the survivor average causal effect