Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation

Abstract

Incorporation of prior knowledge about organ shape and location is key to improve performance of image analysis approaches. In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition. The highly constrained nature of anatomical objects can be well captured with learning based techniques. However, in most recent and promising techniques such as CNN based segmentation it is not obvious how to incorporate such prior knowledge. State-of-the-art methods operate as pixel-wise classifiers where the training objectives do not incorporate the structure and inter-dependencies of the output. To overcome this limitation, we propose a generic training strategy that incorporates anatomical prior knowledge into CNNs through a new regularisation model, which is trained end-to-end. The new framework encourages models to follow the global anatomical properties of the underlying anatomy (e.g. shape, label structure) via learned non-linear representations of the shape. We show that the proposed approach can be easily adapted to different analysis tasks (e.g. image enhancement, segmentation) and improve the prediction accuracy of the state-of-the-art models. The applicability of our approach is shown on multi-modal cardiac datasets and public benchmarks. Additionally, we demonstrate how the learned deep models of 3D shapes can be interpreted and used as biomarkers for classification of cardiac pathologies.

arXiv commentPublished in IEEE Transactions on Medical Imaging (Aug 2017)

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Deep Learning in Medical Image Analysis
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Semantic Segmentation using Adversarial Networks
• A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
• Incorporating prior knowledge in medical image segmentation: a survey
• Iterative Multi-domain Regularized Deep Learning for Anatomical Structure Detection and Segmentation from Ultrasound Images

Cited by in corpus (117)

• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• Deep learning for cardiac image segmentation: A review
• Medical Image Segmentation Using Deep Learning: A Survey
• Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography
• An application of cascaded 3D fully convolutional networks for medical image segmentation
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology
• Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets
• Disentangled Representation Learning in Cardiac Image Analysis
• Attention Enriched Deep Learning Model for Breast Tumor Segmentation in Ultrasound Images
• Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation
• Deep Learning in Cardiology
• Cardiac Segmentation with Strong Anatomical Guarantees
• Learning to Segment from Scribbles using Multi-scale Adversarial Attention Gates
• Automatic Ischemic Stroke Lesion Segmentation from Computed Tomography Perfusion Images by Image Synthesis and Attention-Based Deep Neural Networks
• Pose-Aware Instance Segmentation Framework from Cone Beam CT Images for Tooth Segmentation
• FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
• Learning Deformable Registration of Medical Images with Anatomical Constraints
• Early Detection of Myocardial Infarction in Low-Quality Echocardiography
• Automatic Cardiac Disease Assessment on cine-MRI via Time-Series Segmentation and Domain Specific Features
• Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis
• A Switched View of Retinex: Deep Self-Regularized Low-Light Image Enhancement
• A Distance Map Regularized CNN for Cardiac Cine MR Image Segmentation
• Deep MR Brain Image Super-Resolution Using Spatio-Structural Priors
• Cardiac MRI Segmentation with Strong Anatomical Guarantees
• Segmentation of histological images and fibrosis identification with a convolutional neural network
• Learning With Context Feedback Loop for Robust Medical Image Segmentation
• Cross-dimensional transfer learning in medical image segmentation with deep learning
• A persistent homology-based topological loss for CNN-based multi-class segmentation of CMR
• Learning Shape Priors for Robust Cardiac MR Segmentation from Multi-view Images
• Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
• Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound
• Three-dimensional Cardiovascular Imaging-Genetics: A Mass Univariate Framework
• Lesion Segmentation in Ultrasound Using Semi-pixel-wise Cycle Generative Adversarial Nets
• Annotation-Efficient Learning for Medical Image Segmentation based on Noisy Pseudo Labels and Adversarial Learning
• Medical Imaging with Deep Learning for COVID- 19 Diagnosis: A Comprehensive Review
• Deeply Self-Supervised Contour Embedded Neural Network Applied to Liver Segmentation
• Liver Segmentation in Abdominal CT Images via Auto-Context Neural Network and Self-Supervised Contour Attention
• Unsupervised Deep Contrast Enhancement with Power Constraint for OLED Displays
• Uncertainty Quantification in Deep Learning for Safer Neuroimage Enhancement
• Towards dense volumetric pancreas segmentation in CT using 3D fully convolutional networks
• Segmentation of Bruch's Membrane in retinal OCT with AMD using anatomical priors and uncertainty quantification
• MMGL: Multi-Scale Multi-View Global-Local Contrastive learning for Semi-supervised Cardiac Image Segmentation
• Generalizable multi-task, multi-domain deep segmentation of sparse pediatric imaging datasets via multi-scale contrastive regularization and multi-joint anatomical priors
• Weakly-Supervised 3D Medical Image Segmentation using Geometric Prior and Contrastive Similarity
• Generative adversarial networks and adversarial methods in biomedical image analysis
• Combining Shape Priors with Conditional Adversarial Networks for Improved Scapula Segmentation in MR images
• Multi-structure bone segmentation in pediatric MR images with combined regularization from shape priors and adversarial network
• Linearly Constrained Neural Networks
• Learning Shape Representation on Sparse Point Clouds for Volumetric Image Segmentation
• Unsupervised Domain Adaptation from Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks
• Autoencoding Low-Resolution MRI for Semantically Smooth Interpolation of Anisotropic MRI
• Bottleneck Supervised U-Net for Pixel-wise Liver and Tumor Segmentation
• Star Shape Prior in Fully Convolutional Networks for Skin Lesion Segmentation
• Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Automatic Myocardial Infarction Evaluation from Delayed-Enhancement Cardiac MRI using Deep Convolutional Networks
• Shape constrained CNN for segmentation guided prediction of myocardial shape and pose parameters in cardiac MRI
• Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision
• Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors
• A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
• On orthogonal projections for dimension reduction and applications in augmented target loss functions for learning problems
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Supervision by Denoising for Medical Image Segmentation
• Autoencoders for Multi-Label Prostate MR Segmentation
• Medical image super-resolution method based on dense blended attention network
• VoxelAtlasGAN: 3D Left Ventricle Segmentation on Echocardiography with Atlas Guided Generation and Voxel-to-voxel Discrimination
• Spatio-Temporal Segmentation in 3D Echocardiographic Sequences using Fractional Brownian Motion
• Uncertainty-based graph convolutional networks for organ segmentation refinement
• Automatic Detection of Cardiac Chambers Using an Attention-based YOLOv4 Framework from Four-chamber View of Fetal Echocardiography
• Deep Learning Segmentation in 2D echocardiography using the CAMUS dataset : Automatic Assessment of the Anatomical Shape Validity
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• Leveraging Global Binary Masks for Structure Segmentation in Medical Images
• AtrialJSQnet: A New Framework for Joint Segmentation and Quantification of Left Atrium and Scars Incorporating Spatial and Shape Information
• Semi-overcomplete convolutional auto-encoder embedding as shape priors for deep vessel segmentation
• 3D Consistent & Robust Segmentation of Cardiac Images by Deep Learning with Spatial Propagation
• Target-Embedding Autoencoders for Supervised Representation Learning
• A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation
• Multi-Structure Deep Segmentation with Shape Priors and Latent Adversarial Regularization
• Machine Learning for Automated Mitral Regurgitation Detection from Cardiac Imaging
• Artificial Intelligence-Based Image Reconstruction in Cardiac Magnetic Resonance
• Contour Transformer Network for One-shot Segmentation of Anatomical Structures
• Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity
• Heart Segmentation From MRI Scans Using Convolutional Neural Network
• MIASSR: An Approach for Medical Image Arbitrary Scale Super-Resolution
• Image Segmentation with Topological Priors
• Joint Semi-supervised 3D Super-Resolution and Segmentation with Mixed Adversarial Gaussian Domain Adaptation
• An Alarm System For Segmentation Algorithm Based On Shape Model
• Hybrid graph convolutional neural networks for landmark-based anatomical segmentation
• SHEAR-net: An End-to-End Deep Learning Approach for Single Push Ultrasound Shear Wave Elasticity Imaging
• Quantification of Predictive Uncertainty via Inference-Time Sampling
• Quantifying Topology In Pancreatic Tubular Networks From Live Imaging 3D Microscopy
• An Automatic Cardiac Segmentation Framework based on Multi-sequence MR Image
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Deep Learning with Anatomical Priors: Imitating Enhanced Autoencoders in Latent Space for Improved Pelvic Bone Segmentation in MRI
• Crossover-Net: Leveraging the Vertical-Horizontal Crossover Relation for Robust Segmentation
• Reciprocal Landmark Detection and Tracking with Extremely Few Annotations
• LU-Net: a multi-task network to improve the robustness of segmentation of left ventriclular structures by deep learning in 2D echocardiography
• Labelling Vertebrae with 2D Reformations of Multidetector CT Images: An Adversarial Approach for Incorporating Prior Knowledge of Spine Anatomy
• CNN in CT Image Segmentation: Beyound Loss Function for Expoliting Ground Truth Images
• Single Image Super-resolution via Dense Blended Attention Generative Adversarial Network for Clinical Diagnosis
• Automatically Segmenting the Left Atrium from Cardiac Images Using Successive 3D U-Nets and a Contour Loss
• Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems
• Shape Constrained Network for Eye Segmentation in the Wild
• High-level Prior-based Loss Functions for Medical Image Segmentation: A Survey
• Context-aware virtual adversarial training for anatomically-plausible segmentation
• Joint Motion Correction and Super Resolution for Cardiac Segmentation via Latent Optimisation
• 3D Deep Affine-Invariant Shape Learning for Brain MR Image Segmentation
• EnMcGAN: Adversarial Ensemble Learning for 3D Complete Renal Structures Segmentation
• Sparsely Activated Networks: A new method for decomposing and compressing data
• Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review
• Deep vessel segmentation with joint multi-prior encoding
• Stateless actor-critic for instance segmentation with high-level priors
• Precise Top-Layer Fabric Segmentation for Fabric Destacking with Edge- and Shape-Aware Deep Networks
• Atlas-aware ConvNetfor Accurate yet Robust Anatomical Segmentation
• Shape Constrained CNN for Cardiac MR Segmentation with Simultaneous Prediction of Shape and Pose Parameters