Detect and Correct Bias in Multi-Site Neuroimaging Datasets

Abstract

The desire to train complex machine learning algorithms and to increase the statistical power in association studies drives neuroimaging research to use ever-larger datasets. The most obvious way to increase sample size is by pooling scans from independent studies. However, simple pooling is often ill-advised as selection, measurement, and confounding biases may creep in and yield spurious correlations. In this work, we combine 35,320 magnetic resonance images of the brain from 17 studies to examine bias in neuroimaging. In the first experiment, Name That Dataset, we provide empirical evidence for the presence of bias by showing that scans can be correctly assigned to their respective dataset with 71.5% accuracy. Given such evidence, we take a closer look at confounding bias, which is often viewed as the main shortcoming in observational studies. In practice, we neither know all potential confounders nor do we have data on them. Hence, we model confounders as unknown, latent variables. Kolmogorov complexity is then used to decide whether the confounded or the causal model provides the simplest factorization of the graphical model. Finally, we present methods for dataset harmonization and study their ability to remove bias in imaging features. In particular, we propose an extension of the recently introduced ComBat algorithm to control for global variation across image features, inspired by adjusting for population stratification in genetics. Our results demonstrate that harmonization can reduce dataset-specific information in image features. Further, confounding bias can be reduced and even turned into a causal relationship. However, harmonziation also requires caution as it can easily remove relevant subject-specific information. Code is available at https://github.com/ai-med/Dataset-Bias.

References in corpus (1)

• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo

Cited by in corpus (16)

• How I failed machine learning in medical imaging -- shortcomings and recommendations
• Efficacy of MRI data harmonization in the age of machine learning. A multicenter study across 36 datasets
• Statistical quantification of confounding bias in predictive modelling
• Accelerated functional brain aging in major depressive disorder: evidence from a large scale fMRI analysis of Chinese participants
• A First Look at Dataset Bias in License Plate Recognition
• Federated brain tumor segmentation: an extensive benchmark
• Estimation of Causal Effects in the Presence of Unobserved Confounding in the Alzheimer's Continuum
• Disease-oriented image embedding with pseudo-scanner standardization for content-based image retrieval on 3D brain MRI
• GAN-based generative modelling for dermatological applications -- comparative study
• Towards contrast- and pathology-agnostic clinical fetal brain MRI segmentation using SynthSeg
• Harmonization with Flow-based Causal Inference
• Domain-invariant feature learning in brain MR imaging for content-based image retrieval
• Metadata Normalization
• Embracing the Disharmony in Medical Imaging: A Simple and Effective Framework for Domain Adaptation
• Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning
• What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clustering