BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

Abstract

Recent leading approaches to semantic segmentation rely on deep convolutional networks trained with human-annotated, pixel-level segmentation masks. Such pixel-accurate supervision demands expensive labeling effort and limits the performance of deep networks that usually benefit from more training data. In this paper, we propose a method that achieves competitive accuracy but only requires easily obtained bounding box annotations. The basic idea is to iterate between automatically generating region proposals and training convolutional networks. These two steps gradually recover segmentation masks for improving the networks, and vise versa. Our method, called BoxSup, produces competitive results supervised by boxes only, on par with strong baselines fully supervised by masks under the same setting. By leveraging a large amount of bounding boxes, BoxSup further unleashes the power of deep convolutional networks and yields state-of-the-art results on PASCAL VOC 2012 and PASCAL-CONTEXT.

References in corpus (9)

• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Conditional Random Fields as Recurrent Neural Networks
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Transferring Rich Feature Hierarchies for Robust Visual Tracking
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• Feedforward semantic segmentation with zoom-out features
• Computational Baby Learning

Cited by in corpus (91)

• Fully Convolutional Networks for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Pyramid Scene Parsing Network
• Retinal vessel segmentation based on Fully Convolutional Neural Networks
• Constrained-CNN losses for weakly supervised segmentation
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
• Context Encoding for Semantic Segmentation
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
• Efficient piecewise training of deep structured models for semantic segmentation
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• One-Shot Instance Segmentation
• Fully Convolutional Adaptation Networks for Semantic Segmentation
• RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
• ExFuse: Enhancing Feature Fusion for Semantic Segmentation
• Normalized Cut Loss for Weakly-supervised CNN Segmentation
• Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
• WordSup: Exploiting Word Annotations for Character based Text Detection
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation
• Weakly Supervised Instance Segmentation using Class Peak Response
• COCO-Stuff: Thing and Stuff Classes in Context
• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features
• Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform
• Where are the Masks: Instance Segmentation with Image-level Supervision
• Learning under Distributed Weak Supervision
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
• Simple Does It: Weakly Supervised Instance and Semantic Segmentation
• Exploiting saliency for object segmentation from image level labels
• Weakly Supervised Semantic Segmentation using Web-Crawled Videos
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Deeply Learning the Messages in Message Passing Inference
• Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade
• On Regularized Losses for Weakly-supervised CNN Segmentation
• CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentation
• Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
• LabelBank: Revisiting Global Perspectives for Semantic Segmentation
• Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
• Scene Parsing with Global Context Embedding
• Learning to Segment Every Thing
• Deep Extreme Cut: From Extreme Points to Object Segmentation
• Convolutional Random Walk Networks for Semantic Image Segmentation
• Exploring Context with Deep Structured models for Semantic Segmentation
• Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net
• Multi-level Contextual RNNs with Attention Model for Scene Labeling
• Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST
• Reversible Recursive Instance-level Object Segmentation
• Top-Down Learning for Structured Labeling with Convolutional Pseudoprior
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation
• FoveaNet: Perspective-aware Urban Scene Parsing
• Weakly Labeling the Antarctic: The Penguin Colony Case
• Decoupled Spatial Neural Attention for Weakly Supervised Semantic Segmentation
• Training Group Orthogonal Neural Networks with Privileged Information
• Learning to Segment Medical Images with Scribble-Supervision Alone
• Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer
• Surveillance Video Parsing with Single Frame Supervision
• ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks
• Dense Recurrent Neural Networks for Scene Labeling
• Weakly Supervised Person Re-ID: Differentiable Graphical Learning and A New Benchmark
• Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
• Pseudo Mask Augmented Object Detection
• Learning Semantic Segmentation with Diverse Supervision
• ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation
• SeGAN: Segmenting and Generating the Invisible
• Multi-Miner: Object-Adaptive Region Mining for Weakly-Supervised Semantic Segmentation
• Concept Mask: Large-Scale Segmentation from Semantic Concepts
• Keypoint Based Weakly Supervised Human Parsing
• Hard Pixel Mining for Depth Privileged Semantic Segmentation
• Few-Shot Semantic Segmentation Augmented with Image-Level Weak Annotations
• Learning Deep Representations for Scene Labeling with Semantic Context Guided Supervision
• Semantic Segmentation with Boundary Neural Fields
• Weakly- and Semi-Supervised Panoptic Segmentation
• Importance of Self-Consistency in Active Learning for Semantic Segmentation
• Transferring and Regularizing Prediction for Semantic Segmentation
• Unsupervised Learning of Important Objects from First-Person Videos
• Better Image Segmentation by Exploiting Dense Semantic Predictions
• Recalling Holistic Information for Semantic Segmentation
• Diverse Sampling for Self-Supervised Learning of Semantic Segmentation
• Proposal Flow: Semantic Correspondences from Object Proposals
• Scene Parsing via Dense Recurrent Neural Networks with Attentional Selection
• weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming
• Unsupervised Meta-learning of Figure-Ground Segmentation via Imitating Visual Effects
• Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation
• RethNet: Object-by-Object Learning for Detecting Facial Skin Problems
• Region-based semantic segmentation with end-to-end training
• Learning Pixel-wise Labeling from the Internet without Human Interaction
• WeedMap: A large-scale semantic weed mapping framework using aerial multispectral imaging and deep neural network for precision farming
• Collaborative Annotation of Semantic Objects in Images with Multi-granularity Supervisions
• A Holistic Approach for Data-Driven Object Cutout