Simultaneous Detection and Segmentation
arXiv:1407.1808
Abstract
We aim to detect all instances of a category in an image and, for each instance, mark the pixels that belong to it. We call this task Simultaneous Detection and Segmentation (SDS). Unlike classical bounding box detection, SDS requires a segmentation and not just a box. Unlike classical semantic segmentation, we require individual object instances. We build on recent work that uses convolutional neural networks to classify category-independent region proposals (R-CNN [16]), introducing a novel architecture tailored for SDS. We then use category-specific, top- down figure-ground predictions to refine our bottom-up proposals. We show a 7 point boost (16% relative) over our baselines on SDS, a 5 point boost (10% relative) over state-of-the-art on semantic segmentation, and state-of-the-art performance in object detection. Finally, we provide diagnostic tools that unpack performance and provide directions for future work.
To appear in the European Conference on Computer Vision (ECCV), 2014
References in corpus (1)
Cited by in corpus (100)
- Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
- Fully Convolutional Networks for Semantic Segmentation
- Multiple Instance Learning: A Survey of Problem Characteristics and Applications
- Learning Deconvolution Network for Semantic Segmentation
- Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning
- Convolutional Feature Masking for Joint Object and Stuff Segmentation
- Semantic Instance Segmentation with a Discriminative Loss Function
- Deformable Convolutional Networks
- Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
- Beyond Skip Connections: Top-Down Modulation for Object Detection
- Video (language) modeling: a baseline for generative models of natural videos
- W-Net: A Deep Model for Fully Unsupervised Image Segmentation
- Semantic Instance Segmentation via Deep Metric Learning
- Context Encoding for Semantic Segmentation
- Proposal-free Network for Instance-level Object Segmentation
- IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network
- Locally-Supervised Deep Hybrid Model for Scene Recognition
- Efficient piecewise training of deep structured models for semantic segmentation
- Illuminating Pedestrians via Simultaneous Detection & Segmentation
- Real-World Image Datasets for Federated Learning
- Hide-and-Seek: A Data Augmentation Technique for Weakly-Supervised Localization and Beyond
- Webly Supervised Learning of Convolutional Networks
- Devil in the Details: Towards Accurate Single and Multiple Human Parsing
- Recurrent Neural Networks for Semantic Instance Segmentation
- 3D-BEVIS: Bird's-Eye-View Instance Segmentation
- Learning to Refine Object Segments
- Improving Fast Segmentation With Teacher-student Learning
- Multiple-Human Parsing in the Wild
- Deep convolutional filter banks for texture recognition and segmentation
- What's the Point: Semantic Segmentation with Point Supervision
- Training Deep Networks with Structured Layers by Matrix Backpropagation
- Guided Upsampling Network for Real-Time Semantic Segmentation
- The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks
- COCO-Stuff: Thing and Stuff Classes in Context
- Deep Watershed Transform for Instance Segmentation
- Efficient inference in occlusion-aware generative models of images
- Simple Does It: Weakly Supervised Instance and Semantic Segmentation
- DeepBox: Learning Objectness with Convolutional Networks
- Inferring 3D Object Pose in RGB-D Images
- Instance Segmentation by Deep Coloring
- Convolutional Neural Networks at Constrained Time Cost
- Joint Object and Part Segmentation using Deep Learned Potentials
- ModaNet: A Large-Scale Street Fashion Dataset with Polygon Annotations
- Instance-level Human Parsing via Part Grouping Network
- Semantic Pose using Deep Networks Trained on Synthetic RGB-D
- Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
- Holistic, Instance-Level Human Parsing
- InstanceCut: from Edges to Instances with MultiCut
- Cross Modal Distillation for Supervision Transfer
- Improving Fully Convolution Network for Semantic Segmentation
- Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation
- Hypercolumns for Object Segmentation and Fine-grained Localization
- Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
- Deep GrabCut for Object Selection
- Exploring Context with Deep Structured models for Semantic Segmentation
- Deconvolutional Feature Stacking for Weakly-Supervised Semantic Segmentation
- From Image-level to Pixel-level Labeling with Convolutional Networks
- Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
- Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks
- Training Group Orthogonal Neural Networks with Privileged Information
- Cultural Event Recognition with Visual ConvNets and Temporal Models
- Boundary-aware Instance Segmentation
- One-to-many face recognition with bilinear CNNs
- Semi-convolutional Operators for Instance Segmentation
- BoundarySqueeze: Image Segmentation as Boundary Squeezing
- Deep Feature Factorization For Concept Discovery
- Straight to Shapes: Real-time Detection of Encoded Shapes
- LUCSS: Language-based User-customized Colourization of Scene Sketches
- Pseudo Mask Augmented Object Detection
- Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation
- Outline Objects using Deep Reinforcement Learning
- Learning Rich Features from RGB-D Images for Object Detection and Segmentation
- Semantic Segmentation with Boundary Neural Fields
- Image Labeling with Markov Random Fields and Conditional Random Fields
- Nazr-CNN: Fine-Grained Classification of UAV Imagery for Damage Assessment
- Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
- What's in the box? Explaining the black-box model through an evaluation of its interpretable features
- Weakly- and Semi-Supervised Panoptic Segmentation
- Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation
- Symbolic Segmentation Using Algorithm Selection
- A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation
- Applications of the Streaming Networks
- Semantic Instance Meets Salient Object: Study on Video Semantic Salient Instance Segmentation
- Streaming Networks: Increase Noise Robustness and Filter Diversity via Hard-wired and Input-induced Sparsity
- Complete 3D Scene Parsing from an RGBD Image
- W-Net: Dense Semantic Segmentation of Subcutaneous Tissue in Ultrasound Images by Expanding U-Net to Incorporate Ultrasound RF Waveform Data
- Streaming Networks: Enable A Robust Classification of Noise-Corrupted Images
- Design Pseudo Ground Truth with Motion Cue for Unsupervised Video Object Segmentation
- Unsupervised Video Object Segmentation with Distractor-Aware Online Adaptation
- weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming
- Category-Specific Object Reconstruction from a Single Image
- Affinity Derivation and Graph Merge for Instance Segmentation
- Generating Discriminative Object Proposals via Submodular Ranking
- Region-based semantic segmentation with end-to-end training
- Candidate Constrained CRFs for Loss-Aware Structured Prediction
- Context Based Visual Content Verification
- Sparsifying and Down-scaling Networks to Increase Robustness to Distortions
- MoE-SPNet: A Mixture-of-Experts Scene Parsing Network
- Inner Space Preserving Generative Pose Machine
- Subset Feature Learning for Fine-Grained Category Classification