AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy

Abstract

Methods: Our deep learning model, called AnatomyNet, segments OARs from head and neck CT images in an end-to-end fashion, receiving whole-volume HaN CT images as input and generating masks of all OARs of interest in one shot. AnatomyNet is built upon the popular 3D U-net architecture, but extends it in three important ways: 1) a new encoding scheme to allow auto-segmentation on whole-volume CT images instead of local patches or subsets of slices, 2) incorporating 3D squeeze-and-excitation residual blocks in encoding layers for better feature representation, and 3) a new loss function combining Dice scores and focal loss to facilitate the training of the neural model. These features are designed to address two main challenges in deep-learning-based HaN segmentation: a) segmenting small anatomies (i.e., optic chiasm and optic nerves) occupying only a few slices, and b) training with inconsistent data annotations with missing ground truth for some anatomical structures. Results: We collected 261 HaN CT images to train AnatomyNet, and used MICCAI Head and Neck Auto Segmentation Challenge 2015 as a benchmark dataset to evaluate the performance of AnatomyNet. The objective is to segment nine anatomies: brain stem, chiasm, mandible, optic nerve left, optic nerve right, parotid gland left, parotid gland right, submandibular gland left, and submandibular gland right. Compared to previous state-of-the-art results from the MICCAI 2015 competition, AnatomyNet increases Dice similarity coefficient by 3.3% on average. AnatomyNet takes about 0.12 seconds to fully segment a head and neck CT image of dimension 178 x 302 x 225, significantly faster than previous methods. In addition, the model is able to process whole-volume CT images and delineate all OARs in one pass, requiring little pre- or post-processing. https://github.com/wentaozhu/AnatomyNet-for-anatomical-segmentation.git.

6 figures, 4 videos in GitHub and YouTube. Accepted by Medical Physics. Code and videos are available on GitHub. Video: https://www.youtube.com/watch?v=_PpIUIm4XLU

References in corpus (4)

• Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
• Squeeze-and-Excitation Networks
• Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy
• Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification

Cited by in corpus (51)

• Deep Learning System to Screen Coronavirus Disease 2019 Pneumonia
• Boundary loss for highly unbalanced segmentation
• Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy
• Attention Enriched Deep Learning Model for Breast Tumor Segmentation in Ultrasound Images
• FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
• A Deep Learning System That Generates Quantitative CT Reports for Diagnosing Pulmonary Tuberculosis
• Automation of Radiation Treatment Planning for Rectal Cancer
• CASI: A Convolutional Neural Network Approach for Shell Identification
• Weaving Attention U-net: A Novel Hybrid CNN and Attention-based Method for Organs-at-risk Segmentation in Head and Neck CT Images
• Recalibrating 3D ConvNets with Project & Excite
• Precise Estimation of Renal Vascular Dominant Regions Using Spatially Aware Fully Convolutional Networks, Tensor-Cut and Voronoi Diagrams
• External Attention Assisted Multi-Phase Splenic Vascular Injury Segmentation with Limited Data
• DeepHealth: Review and challenges of artificial intelligence in health informatics
• blob loss: instance imbalance aware loss functions for semantic segmentation
• Registration-Guided Deep Learning Image Segmentation for Cone Beam CT-based Online Adaptive Radiotherapy
• Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• Deep Dilated Convolutional Nets for the Automatic Segmentation of Retinal Vessels
• Shape-Aware Organ Segmentation by Predicting Signed Distance Maps
• Saliency-Guided Deep Learning Network for Automatic Tumor Bed Volume Delineation in Post-operative Breast Irradiation
• Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation
• Modality-aware Mutual Learning for Multi-modal Medical Image Segmentation
• A Combined Deep Learning-Gradient Boosting Machine Framework for Fluid Intelligence Prediction
• `Project & Excite' Modules for Segmentation of Volumetric Medical Scans
• FAS-UNet: A Novel FAS-driven Unet to Learn Variational Image Segmentation
• Tempera: Spatial Transformer Feature Pyramid Network for Cardiac MRI Segmentation
• LAMP: Large Deep Nets with Automated Model Parallelism for Image Segmentation
• Organ at Risk Segmentation in Head and Neck CT Images by Using a Two-Stage Segmentation Framework Based on 3D U-Net
• A fully 3D multi-path convolutional neural network with feature fusion and feature weighting for automatic lesion identification in brain MRI images
• Deep Learning in Multi-organ Segmentation
• AV-Net: Deep learning for fully automated artery-vein classification in optical coherence tomography angiography
• NeurReg: Neural Registration and Its Application to Image Segmentation
• Automatic Segmentation of Organs-at-Risk from Head-and-Neck CT using Separable Convolutional Neural Network with Hard-Region-Weighted Loss
• Test-Time Training for Deformable Multi-Scale Image Registration
• Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation
• Automatic Pulmonary Lobe Segmentation Using Deep Learning
• Deformable Gabor Feature Networks for Biomedical Image Classification
• Anatomically Consistent Segmentation of Organs at Risk in MRI with Convolutional Neural Networks
• Deep Learning for Automated Medical Image Analysis
• Multi-Scale Attentional Network for Multi-Focal Segmentation of Active Bleed after Pelvic Fractures
• Recurrent convolutional neural networks for mandible segmentation from computed tomography
• OARnet: Automated organs-at-risk delineation in Head and Neck CT images
• AttentionAnatomy: A unified framework for whole-body organs at risk segmentation using multiple partially annotated datasets
• Mining Domain Knowledge: Improved Framework towards Automatically Standardizing Anatomical Structure Nomenclature in Radiotherapy
• DeepWL: Robust EPID based Winston-Lutz Analysis using Deep Learning and Synthetic Image Generation
• FDA: Feature Decomposition and Aggregation for Robust Airway Segmentation
• A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images
• Gray Matter Segmentation in Ultra High Resolution 7 Tesla ex vivo T2w MRI of Human Brain Hemispheres
• Automatic Segmentation of Gross Target Volume of Nasopharynx Cancer using Ensemble of Multiscale Deep Neural Networks with Spatial Attention
• Polynomial Dice Loss for Medical Image Segmentation
• Cardiac Segmentation of LGE MRI with Noisy Labels