Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects

Abstract

This is an empirical study to investigate the impact of scanner effects when using machine learning on multi-site neuroimaging data. We utilize structural T1-weighted brain MRI obtained from two different studies, Cam-CAN and UK Biobank. For the purpose of our investigation, we construct a dataset consisting of brain scans from 592 age- and sex-matched individuals, 296 subjects from each original study. Our results demonstrate that even after careful pre-processing with state-of-the-art neuroimaging pipelines a classifier can easily distinguish between the origin of the data with very high accuracy. Our analysis on the example application of sex classification suggests that current approaches to harmonize data are unable to remove scanner-specific bias leading to overly optimistic performance estimates and poor generalization. We conclude that multi-site data harmonization remains an open challenge and particular care needs to be taken when using such data with advanced machine learning methods for predictive modelling.

Presented at the Medical Imaging meets NeurIPS Workshop 2019

Cited by in corpus (10)

• The Liver Tumor Segmentation Benchmark (LiTS)
• Contrastive Cross-site Learning with Redesigned Net for COVID-19 CT Classification
• Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research
• Multi-Center Fetal Brain Tissue Annotation (FeTA) Challenge 2022 Results
• Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks
• Domain Adaptive Medical Image Segmentation via Adversarial Learning of Disease-Specific Spatial Patterns
• Harmonization and the Worst Scanner Syndrome
• Causality-inspired Single-source Domain Generalization for Medical Image Segmentation
• Robust White Matter Hyperintensity Segmentation on Unseen Domain
• Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning