Constrained-CNN losses for weakly supervised segmentation

Abstract

Weakly-supervised learning based on, e.g., partially labelled images or image-tags, is currently attracting significant attention in CNN segmentation as it can mitigate the need for full and laborious pixel/voxel annotations. Enforcing high-order (global) inequality constraints on the network output (for instance, to constrain the size of the target region) can leverage unlabeled data, guiding the training process with domain-specific knowledge. Inequality constraints are very flexible because they do not assume exact prior knowledge. However, constrained Lagrangian dual optimization has been largely avoided in deep networks, mainly for computational tractability reasons. To the best of our knowledge, the method of [Pathak et al., 2015] is the only prior work that addresses deep CNNs with linear constraints in weakly supervised segmentation. It uses the constraints to synthesize fully-labeled training masks (proposals) from weak labels, mimicking full supervision and facilitating dual optimization. We propose to introduce a differentiable penalty, which enforces inequality constraints directly in the loss function, avoiding expensive Lagrangian dual iterates and proposal generation. From constrained-optimization perspective, our simple penalty-based approach is not optimal as there is no guarantee that the constraints are satisfied. However, surprisingly, it yields substantially better results than the Lagrangian-based constrained CNNs in [Pathak et al., 2015], while reducing the computational demand for training. By annotating only a small fraction of the pixels, the proposed approach can reach a level of segmentation performance that is comparable to full supervision on three separate tasks. While our experiments focused on basic linear constraints such as the target-region size and image tags, our framework can be easily extended to other non-linear constraints.

arXiv commentExtended work of the work presented at the 1st conference on Medical Image with Deep Learning (MIDL). Currently under review at Medical Image Analysis

References in corpus (16)

• A Survey on Deep Learning in Medical Image Analysis
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study
• FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics
• Fully Convolutional Multi-Class Multiple Instance Learning
• Constrained Deep Weak Supervision for Histopathology Image Segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Imposing Hard Constraints on Deep Networks: Promises and Limitations
• Normalized Cut Loss for Weakly-supervised CNN Segmentation
• Learning under Distributed Weak Supervision
• Semi-Supervised Multi-Organ Segmentation via Deep Multi-Planar Co-Training
• Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision

Cited by in corpus (45)

• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• Deep learning for cardiac image segmentation: A review
• Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
• Learning to Segment from Scribbles using Multi-scale Adversarial Attention Gates
• Automatic Ischemic Stroke Lesion Segmentation from Computed Tomography Perfusion Images by Image Synthesis and Attention-Based Deep Neural Networks
• A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains
• ExplAIn: Explanatory Artificial Intelligence for Diabetic Retinopathy Diagnosis
• Curriculum semi-supervised segmentation
• Constrained Deep Networks: Lagrangian Optimization via Log-Barrier Extensions
• Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Annotation-Efficient Learning for Medical Image Segmentation based on Noisy Pseudo Labels and Adversarial Learning
• A Survey on Deep Learning of Small Sample in Biomedical Image Analysis
• Constrained domain adaptation for Image segmentation
• Revisiting CycleGAN for semi-supervised segmentation
• Segmentation with mixed supervision: Confidence maximization helps knowledge distillation
• Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation
• Weakly-Supervised 3D Medical Image Segmentation using Geometric Prior and Contrastive Similarity
• Weakly supervised segmentation from extreme points
• An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization
• Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision
• Anatomy-guided domain adaptation for 3D in-bed human pose estimation
• Weakly-supervised Learning For Catheter Segmentation in 3D Frustum Ultrasound
• Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis
• Clinical Domain Knowledge-Derived Template Improves Post Hoc AI Explanations in Pneumothorax Classification
• Automatic cerebral hemisphere segmentation in rat MRI with lesions via attention-based convolutional neural networks
• Min-max Entropy for Weakly Supervised Pointwise Localization
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• Recent Advances in Fibrosis and Scar Segmentation from Cardiac MRI: A State-of-the-Art Review and Future Perspectives
• Machine Learning for Automated Mitral Regurgitation Detection from Cardiac Imaging
• Mixed-Supervised Dual-Network for Medical Image Segmentation
• Discretely-constrained deep network for weakly supervised segmentation
• Learning to segment prostate cancer by aggressiveness from scribbles in bi-parametric MRI
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• LCollision: Fast Generation of Collision-Free Human Poses using Learned Non-Penetration Constraints
• Point-to-set distance functions for weakly supervised segmentation
• Stateless actor-critic for instance segmentation with high-level priors
• One-shot Weakly-Supervised Segmentation in Medical Images
• Constrained unsupervised anomaly segmentation
• Context-aware virtual adversarial training for anatomically-plausible segmentation
• High-level Prior-based Loss Functions for Medical Image Segmentation: A Survey
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation
• Atlas-aware ConvNetfor Accurate yet Robust Anatomical Segmentation
• A Weakly-Supervised Semantic Segmentation Approach based on the Centroid Loss: Application to Quality Control and Inspection