Learning joint segmentation of tissues and brain lesions from task-specific hetero-modal domain-shifted datasets

Abstract

Brain tissue segmentation from multimodal MRI is a key building block of many neuroimaging analysis pipelines. Established tissue segmentation approaches have, however, not been developed to cope with large anatomical changes resulting from pathology, such as white matter lesions or tumours, and often fail in these cases. In the meantime, with the advent of deep neural networks (DNNs), segmentation of brain lesions has matured significantly. However, few existing approaches allow for the joint segmentation of normal tissue and brain lesions. Developing a DNN for such a joint task is currently hampered by the fact that annotated datasets typically address only one specific task and rely on task-specific imaging protocols including a task-specific set of imaging modalities. In this work, we propose a novel approach to build a joint tissue and lesion segmentation model from aggregated task-specific hetero-modal domain-shifted and partially-annotated datasets. Starting from a variational formulation of the joint problem, we show how the expected risk can be decomposed and optimised empirically. We exploit an upper bound of the risk to deal with heterogeneous imaging modalities across datasets. To deal with potential domain shift, we integrated and tested three conventional techniques based on data augmentation, adversarial learning and pseudo-healthy generation. For each individual task, our joint approach reaches comparable performance to task-specific and fully-supervised models. The proposed framework is assessed on two different types of brain lesions: White matter lesions and gliomas. In the latter case, lacking a joint ground-truth for quantitative assessment purposes, we propose and use a novel clinically-relevant qualitative assessment methodology.

arXiv commentMIDL 2019 special issue - Medical Image Analysis

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Instance Normalization: The Missing Ingredient for Fast Stylization
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Learning Transferable Features with Deep Adaptation Networks
• An Overview of Multi-Task Learning in Deep Neural Networks
• Conditional Adversarial Domain Adaptation
• A large annotated medical image dataset for the development and evaluation of segmentation algorithms
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
• A Survey on Multi-Task Learning
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities
• Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge
• On Learning Invariant Representation for Domain Adaptation
• Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation
• Automatic CAD-RADS Scoring Using Deep Learning
• Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders
• Multitask Classification and Segmentation for Cancer Diagnosis in Mammography
• An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection
• Adversarial Pseudo Healthy Synthesis Needs Pathology Factorization
• Joint Learning of Brain Lesion and Anatomy Segmentation from Heterogeneous Datasets
• Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks

Cited by in corpus (9)

• Can segmentation models be trained with fully synthetically generated data?
• Label-set Loss Functions for Partial Supervision: Application to Fetal Brain 3D MRI Parcellation
• A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections
• Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations
• Multitask Learning in Minimally Invasive Surgical Vision: A Review
• Multi-center anatomical segmentation with heterogeneous labels via landmark-based models
• Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts
• Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift
• Deep Biomechanically-Guided Interpolation for Keypoint-Based Brain Shift Registration