Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields

Abstract

Automatic segmentation of the liver and its lesion is an important step towards deriving quantitative biomarkers for accurate clinical diagnosis and computer-aided decision support systems. This paper presents a method to automatically segment liver and lesions in CT abdomen images using cascaded fully convolutional neural networks (CFCNs) and dense 3D conditional random fields (CRFs). We train and cascade two FCNs for a combined segmentation of the liver and its lesions. In the first step, we train a FCN to segment the liver as ROI input for a second FCN. The second FCN solely segments lesions from the predicted liver ROIs of step 1. We refine the segmentations of the CFCN using a dense 3D CRF that accounts for both spatial coherence and appearance. CFCN models were trained in a 2-fold cross-validation on the abdominal CT dataset 3DIRCAD comprising 15 hepatic tumor volumes. Our results show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over 94% for liver with computation times below 100s per volume. We experimentally demonstrate the robustness of the proposed method as a decision support system with a high accuracy and speed for usage in daily clinical routine.

Accepted at MICCAI 2016. Source code available on https://github.com/IBBM/Cascaded-FCN

References in corpus (9)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Brain Tumor Segmentation with Deep Neural Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Fully Convolutional Networks for Semantic Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
• Metastatic liver tumour segmentation from discriminant Grassmannian manifolds

Cited by in corpus (61)

• A Survey on Deep Learning in Medical Image Analysis
• MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation
• The Liver Tumor Segmentation Benchmark (LiTS)
• Medical Image Segmentation Using Deep Learning: A Survey
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
• An application of cascaded 3D fully convolutional networks for medical image segmentation
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Modality specific U-Net variants for biomedical image segmentation: A survey
• Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer
• Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection
• Efficient Medical Image Segmentation Based on Knowledge Distillation
• Mumford-Shah Loss Functional for Image Segmentation with Deep Learning
• Semantic Segmentation of Pathological Lung Tissue with Dilated Fully Convolutional Networks
• Hierarchical 3D fully convolutional networks for multi-organ segmentation
• FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
• Deep Sequential Segmentation of Organs in Volumetric Medical Scans
• A Method of Rapid Quantification of Patient-Specific Organ Dose for CT Using Coupled Deep-Learning based Multi-Organ Segmentation and GPU-accelerated Monte Carlo Dose Computing
• Multi-region segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks
• PA-ResSeg: A Phase Attention Residual Network for Liver Tumor Segmentation from Multi-phase CT Images
• Volumetric Attention for 3D Medical Image Segmentation and Detection
• Deep convolutional neural networks for multi-planar lung nodule detection: improvement in small nodule identification
• Precise Estimation of Renal Vascular Dominant Regions Using Spatially Aware Fully Convolutional Networks, Tensor-Cut and Voronoi Diagrams
• DeepHealth: Review and challenges of artificial intelligence in health informatics
• Segmentation of Liver Lesions with Reduced Complexity Deep Models
• Robust End-to-End Focal Liver Lesion Detection using Unregistered Multiphase Computed Tomography Images
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps
• Anatomical Data Augmentation For CNN based Pixel-wise Classification
• SurvivalNet: Predicting patient survival from diffusion weighted magnetic resonance images using cascaded fully convolutional and 3D convolutional neural networks
• DINs: Deep Interactive Networks for Neurofibroma Segmentation in Neurofibromatosis Type 1 on Whole-Body MRI
• Feature Fusion Encoder Decoder Network For Automatic Liver Lesion Segmentation
• Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation
• DARWIN: A Highly Flexible Platform for Imaging Research in Radiology
• A Bayesian Hierarchical Network for Combining Heterogeneous Data Sources in Medical Diagnoses
• A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation
• A Model-Driven Stack-Based Fully Convolutional Network for Pancreas Segmentation
• A Multiscale Patch Based Convolutional Network for Brain Tumor Segmentation
• Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
• Deep Learning in Multi-organ Segmentation
• SAR-U-Net: squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in Computed Tomography
• Small Organ Segmentation in Whole-body MRI using a Two-stage FCN and Weighting Schemes
• A comparative study of source-finding techniques in HI emission line cubes using SoFiA, MTObjects, and supervised deep learning
• Context-aware PolyUNet for Liver and Lesion Segmentation from Abdominal CT Images
• Towards Automatic 3D Shape Instantiation for Deployed Stent Grafts: 2D Multiple-class and Class-imbalance Marker Segmentation with Equally-weighted Focal U-Net
• Multi-phase Liver Tumor Segmentation with Spatial Aggregation and Uncertain Region Inpainting
• TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors
• Perturb-and-MPM: Quantifying Segmentation Uncertainty in Dense Multi-Label CRFs
• Ensemble Methods for Multi-Organ Segmentation in CT Series
• Structurally aware bidirectional unpaired image to image translation between CT and MR
• Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation
• CT Image Enhancement Using Stacked Generative Adversarial Networks and Transfer Learning for Lesion Segmentation Improvement
• A New Three-stage Curriculum Learning Approach to Deep Network Based Liver Tumor Segmentation
• Tracking-Assisted Segmentation of Biological Cells
• Boosted Training of Convolutional Neural Networks for Multi-Class Segmentation
• Semantic Feature Attention Network for Liver Tumor Segmentation in Large-scale CT database
• Liver Segmentation in Time-resolved C-arm CT Volumes Reconstructed from Dynamic Perfusion Scans using Time Separation Technique
• Scribble-based Hierarchical Weakly Supervised Learning for Brain Tumor Segmentation
• CleftNet: Augmented Deep Learning for Synaptic Cleft Detection from Brain Electron Microscopy
• Sparsely Activated Networks: A new method for decomposing and compressing data
• Detecting and Segmenting Adversarial Graphics Patterns from Images