1.6k citations · 3.3k across the 24 of their papers we have counts for
7 papers · 1 filter
Beyond algorithm hyperparameters: on preprocessing hyperparameters and associated pitfalls in machine learning applications
Christina Sauer, Anne-Laure Boulesteix, Luzia Hanßum +3
Adequately generating and evaluating prediction models based on supervised machine learning (ML) is often challenging, especially for less experienced users in applied research are…
Constructing Confidence Intervals for 'the' Generalization Error -- a Comprehensive Benchmark Study
Hannah Schulz-Kümpel, Sebastian Fischer, Roman Hornung +3
When assessing the quality of prediction models in machine learning, confidence intervals (CIs) for the generalization error, which measures predictive performance, are a crucial t…
Rethinking the handling of method failure in comparison studies
Milena Wünsch, Moritz Herrmann, Elisa Noltenius +3
Comparison studies in methodological research are intended to compare methods in an evidence-based manner to help data analysts select a suitable method for their application. To p…
Position: Why We Must Rethink Empirical Research in Machine Learning
Moritz Herrmann, F. Julian D. Lange, Katharina Eggensperger +7
We warn against a common but incomplete understanding of empirical research in machine learning that leads to non-replicable results, makes findings unreliable, and threatens to un…
Understanding overfitting in random forest for probability estimation: a visualization and simulation study
Lasai Barreñada, Paula Dhiman, Dirk Timmerman +2
Random forests have become popular for clinical risk prediction modelling. In a case study on predicting ovarian malignancy, we observed training c-statistics close to 1. Although…
To tweak or not to tweak. How exploiting flexibilities in gene set analysis leads to over-optimism
Milena Wünsch, Christina Sauer, Moritz Herrmann +2
Gene set analysis, a popular approach for analysing high-throughput gene expression data, aims to identify sets of genes that show enriched expression patterns between two conditio…
Addressing researcher degrees of freedom through minP adjustment
Maximilian M Mandl, Andrea S Becker-Pennrich, Ludwig C Hinske +2
When different researchers study the same research question using the same dataset they may obtain different and potentially even conflicting results. This is because there is ofte…