Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

Abstract

Deep convolutional neural networks (DCNNs) trained on a large number of images with strong pixel-level annotations have recently significantly pushed the state-of-art in semantic image segmentation. We study the more challenging problem of learning DCNNs for semantic image segmentation from either (1) weakly annotated training data such as bounding boxes or image-level labels or (2) a combination of few strongly labeled and many weakly labeled images, sourced from one or multiple datasets. We develop Expectation-Maximization (EM) methods for semantic image segmentation model training under these weakly supervised and semi-supervised settings. Extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentation benchmark, while requiring significantly less annotation effort. We share source code implementing the proposed system at https://bitbucket.org/deeplab/deeplab-public.

Accepted to ICCV 2015

References in corpus (11)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Fully Convolutional Networks for Semantic Segmentation
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Feedforward semantic segmentation with zoom-out features
• Untangling Local and Global Deformations in Deep Convolutional Networks for Image Classification and Sliding Window Detection
• Hypercolumns for Object Segmentation and Fine-grained Localization
• Material Recognition in the Wild with the Materials in Context Database

Cited by in corpus (107)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Conditional Random Fields as Recurrent Neural Networks
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Learning Deconvolution Network for Semantic Segmentation
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
• Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro
• Semantic Image Segmentation via Deep Parsing Network
• Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Image Segmentation Algorithms Overview
• CrowdNet: A Deep Convolutional Network for Dense Crowd Counting
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
• Proposal-free Network for Instance-level Object Segmentation
• ExplAIn: Explanatory Artificial Intelligence for Diabetic Retinopathy Diagnosis
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Curriculum semi-supervised segmentation
• Makeup like a superstar: Deep Localized Makeup Transfer Network
• Adversarial Complementary Learning for Weakly Supervised Object Localization
• Tell Me Where to Look: Guided Attention Inference Network
• Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation
• A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects
• Weak Supervision for Generating Pixel-Level Annotations in Scene Text Segmentation
• Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation
• Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations
• Webly Supervised Learning of Convolutional Networks
• One-Shot Instance Segmentation
• Listening for Sirens: Locating and Classifying Acoustic Alarms in City Scenes
• Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
• Pushing the Boundaries of Boundary Detection using Deep Learning
• Looking at Outfit to Parse Clothing
• What's the Point: Semantic Segmentation with Point Supervision
• Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
• Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach
• Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation
• Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation
• Learning from Pixel-Level Label Noise: A New Perspective for Semi-Supervised Semantic Segmentation
• Weakly Supervised Instance Segmentation using Class Peak Response
• Sentence-State LSTM for Text Representation
• Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform
• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• Learning under Distributed Weak Supervision
• Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification
• Face Parsing via a Fully-Convolutional Continuous CRF Neural Network
• Simple Does It: Weakly Supervised Instance and Semantic Segmentation
• Exploiting saliency for object segmentation from image level labels
• Monocular Depth Estimation using Multi-Scale Continuous CRFs as Sequential Deep Networks
• Weakly Supervised Semantic Segmentation using Web-Crawled Videos
• Revisiting CycleGAN for semi-supervised segmentation
• Deeply Learning the Messages in Message Passing Inference
• Weakly and Semi Supervised Detection in Medical Imaging via Deep Dual Branch Net
• Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation
• Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach
• Deep Learning Markov Random Field for Semantic Segmentation
• Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation
• Signet Ring Cell Detection With a Semi-supervised Learning Framework
• Self-Erasing Network for Integral Object Attention
• Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation
• Learning to Segment Every Thing
• Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation
• Improved Part Segmentation Performance by Optimising Realism of Synthetic Images using Cycle Generative Adversarial Networks
• Deconvolutional Feature Stacking for Weakly-Supervised Semantic Segmentation
• Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks
• Deep Co-Training for Semi-Supervised Image Segmentation
• Inter-Image Communication for Weakly Supervised Localization
• Weakly-Supervised Semantic Segmentation via Sub-category Exploration
• Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network
• Employing Weak Annotations for Medical Image Analysis Problems
• Weakly Labeling the Antarctic: The Penguin Colony Case
• Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation
• Deep video gesture recognition using illumination invariants
• Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy
• Fast Semantic Image Segmentation with High Order Context and Guided Filtering
• WebSeg: Learning Semantic Segmentation from Web Searches
• Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer
• DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks
• ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks
• Weakly Supervised Person Re-ID: Differentiable Graphical Learning and A New Benchmark
• Partial Labeled Gastric Tumor Segmentation via patch-based Reiterative Learning
• Cumulo: A Dataset for Learning Cloud Classes
• Learning Semantic Segmentation with Diverse Supervision
• SeGAN: Segmenting and Generating the Invisible
• Keypoint Based Weakly Supervised Human Parsing
• Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues
• Top-down Neural Attention by Excitation Backprop
• Adversarial Constraint Learning for Structured Prediction
• LID 2020: The Learning from Imperfect Data Challenge Results
• Manifold-driven Attention Maps for Weakly Supervised Segmentation
• Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
• Weakly- and Semi-Supervised Panoptic Segmentation
• Inverting Variational Autoencoders for Improved Generative Accuracy
• Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation
• Unsupervised Learning of Important Objects from First-Person Videos
• DAF-NET: a saliency based weakly supervised method of dual attention fusion for fine-grained image classification
• A Holistic Approach for Data-Driven Object Cutout
• Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
• Learning to segment with image-level supervision
• Towards well-specified semi-supervised model-based classifiers via structural adaptation
• Realizing Pixel-Level Semantic Learning in Complex Driving Scenes based on Only One Annotated Pixel per Class
• A Brief Survey and an Application of Semantic Image Segmentation for Autonomous Driving
• Closed-Loop Adaptation for Weakly-Supervised Semantic Segmentation
• Learning Pixel-wise Labeling from the Internet without Human Interaction
• Fully Convolutional Neural Networks for Dynamic Object Detection in Grid Maps (Masters Thesis)