1.6k citations · 3.3k across the 24 of their papers we have counts for
28 papers
Statistical parametric simulation studies based on real data
Christina Sauer, F. Julian D. Lange, Maria Thurow +2
Simulation studies are indispensable for evaluating statistical methods and ubiquitous in statistical research. The most common simulation approach is parametric simulation, where…
The impact of the storytelling fallacy on real data examples in methodological research
Maximilian M. Mandl, Frank Weber, Tobias Wöhrle +1
The term "researcher degrees of freedom" (RDF), which was introduced in metascientific literature in the context of the replication crisis in science, refers to the extent of flexi…
On "Confirmatory" Methodological Research in Statistics and Related Fields
F. J. D. Lange, Juliane C. Wilcke, Sabine Hoffmann +2
Empirical substantive research, such as in the life or social sciences, is commonly categorized into the two modes exploratory and confirmatory, both of which are essential to scie…
Outlier Detection in Mendelian Randomisation
Maximilian M Mandl, Anne-Laure Boulesteix, Stephen Burgess +1
Mendelian Randomisation (MR) uses genetic variants as instrumental variables to infer causal effects of exposures on an outcome. One key assumption of MR is that the genetic varian…
Beyond algorithm hyperparameters: on preprocessing hyperparameters and associated pitfalls in machine learning applications
Christina Sauer, Anne-Laure Boulesteix, Luzia Hanßum +3
Adequately generating and evaluating prediction models based on supervised machine learning (ML) is often challenging, especially for less experienced users in applied research are…
Constructing Confidence Intervals for 'the' Generalization Error -- a Comprehensive Benchmark Study
Hannah Schulz-Kümpel, Sebastian Fischer, Roman Hornung +3
When assessing the quality of prediction models in machine learning, confidence intervals (CIs) for the generalization error, which measures predictive performance, are a crucial t…
Rethinking the handling of method failure in comparison studies
Milena Wünsch, Moritz Herrmann, Elisa Noltenius +3
Comparison studies in methodological research are intended to compare methods in an evidence-based manner to help data analysts select a suitable method for their application. To p…
Position: Why We Must Rethink Empirical Research in Machine Learning
Moritz Herrmann, F. Julian D. Lange, Katharina Eggensperger +7
We warn against a common but incomplete understanding of empirical research in machine learning that leads to non-replicable results, makes findings unreliable, and threatens to un…
Understanding overfitting in random forest for probability estimation: a visualization and simulation study
Lasai Barreñada, Paula Dhiman, Dirk Timmerman +2
Random forests have become popular for clinical risk prediction modelling. In a case study on predicting ovarian malignancy, we observed training c-statistics close to 1. Although…
To tweak or not to tweak. How exploiting flexibilities in gene set analysis leads to over-optimism
Milena Wünsch, Christina Sauer, Moritz Herrmann +2
Gene set analysis, a popular approach for analysing high-throughput gene expression data, aims to identify sets of genes that show enriched expression patterns between two conditio…
Addressing researcher degrees of freedom through minP adjustment
Maximilian M Mandl, Andrea S Becker-Pennrich, Ludwig C Hinske +2
When different researchers study the same research question using the same dataset they may obtain different and potentially even conflicting results. This is because there is ofte…
Evaluating machine learning models in non-standard settings: An overview and new findings
Roman Hornung, Malte Nalenz, Lennart Schneider +5
Estimating the generalization error (GE) of machine learning models is fundamental, with resampling methods being the most common approach. However, in non-standard settings, parti…
From RNA sequencing measurements to the final results: a practical guide to navigating the choices and uncertainties of gene set analysis
Milena Wünsch, Christina Sauer, Patrick Callahan +2
Gene set analysis, a popular approach for analyzing high-throughput gene expression data, aims to identify sets of related genes that show significantly enriched or depleted expres…
Prediction approaches for partly missing multi-omics covariate data: A literature review and an empirical comparison study
Roman Hornung, Frederik Ludwigs, Jonas Hagenberg +1
As the availability of omics data has increased in the last few years, more multi-omics data have been generated, that is, high-dimensional molecular data consisting of several typ…
Phases of methodological research in biostatistics - building the evidence base for new methods
Georg Heinze, Anne-Laure Boulesteix, Michael Kammer +2
Although the biostatistical scientific literature publishes new methods at a very high rate, many of these developments are not trustworthy enough to be adopted by the scientific c…
Explaining the optimistic performance evaluation of newly proposed methods: a cross-design validation experiment
Christina Nießl, Sabine Hoffmann, Theresa Ullmann +1
The constant development of new data analysis methods in many fields of research is accompanied by an increasing awareness that these new methods often perform better in their intr…
Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges
Bernd Bischl, Martin Binder, Michel Lang +9
Most machine learning algorithms are configured by one or several hyperparameters that must be carefully chosen and often considerably impact performance. To avoid a time consuming…
Validation of cluster analysis results on validation data: A systematic framework
Theresa Ullmann, Christian Hennig, Anne-Laure Boulesteix
Cluster analysis refers to a wide range of data analytic techniques for class discovery and is popular in many application fields. To judge the quality of a clustering result, diff…
Large-scale benchmark study of survival prediction methods using multi-omics data
Moritz Herrmann, Philipp Probst, Roman Hornung +2
Multi-omics data, that is, datasets containing different types of high-dimensional molecular variables (often in addition to classical clinical variables), are increasingly generat…
Essential guidelines for computational method benchmarking
Lukas M. Weber, Wouter Saelens, Robrecht Cannoodt +6
In computational biology and other sciences, researchers are frequently faced with a choice between several computational methods for performing data analyses. Benchmarking studies…
Benchmarking in cluster analysis: A white paper
Iven Van Mechelen, Anne-Laure Boulesteix, Rainer Dangl +5
Note: A revised version of this is now published. Please cite and read (it's open access): Van Mechelen, I., Boulesteix, A.-L., Dangl, R., Dean, N., Hennig, C., Leisch, F., Steinle…
Hyperparameters and Tuning Strategies for Random Forest
Philipp Probst, Marvin Wright, Anne-Laure Boulesteix
The random forest algorithm (RF) has several hyperparameters that have to be set by the user, e.g., the number of observations drawn randomly for each tree and whether they are dra…
Tunability: Importance of Hyperparameters of Machine Learning Algorithms
Philipp Probst, Bernd Bischl, Anne-Laure Boulesteix
Modern supervised machine learning algorithms involve hyperparameters that have to be set before running them. Options for setting hyperparameters are default values from the softw…
To tune or not to tune the number of trees in random forest?
Philipp Probst, Anne-Laure Boulesteix
The number of trees T in the random forest (RF) algorithm for supervised learning has to be set by the user. It is controversial whether T should simply be set to the largest compu…