Learning to Segment Object Candidates

Abstract

Recent object detection systems rely on two critical steps: (1) a set of object proposals is predicted as efficiently as possible, and (2) this set of candidate proposals is then passed to an object classifier. Such approaches have been shown they can be fast, while achieving the state of the art in detection performance. In this paper, we propose a new way to generate object proposals, introducing an approach based on a discriminative convolutional network. Our model is trained jointly with two objectives: given an image patch, the first part of the system outputs a class-agnostic segmentation mask, while the second part of the system outputs the likelihood of the patch being centered on a full object. At test time, the model is efficiently applied on the whole test image and generates a set of segmentation masks, each of them being assigned with a corresponding object likelihood score. We show that our model yields significant improvements over state-of-the-art object proposal algorithms. In particular, compared to previous approaches, our model obtains substantially higher object recall using fewer proposals. We also show that our model is able to generalize to unseen categories it has not seen during training. Unlike all previous approaches for generating object masks, we do not rely on edges, superpixels, or any other form of low-level segmentation.

References in corpus (5)

• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• What makes for effective detection proposals?
• Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
• Scalable, High-Quality Object Detection
• DeepBox: Learning Objectness with Convolutional Networks

Cited by in corpus (160)

• Focal Loss for Dense Object Detection
• Explainable Machine Learning for Scientific Insights and Discoveries
• A Review on Deep Learning Techniques Applied to Semantic Segmentation
• What makes for effective detection proposals?
• DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
• Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Feature Pyramid Networks for Object Detection
• DenseBox: Unifying Landmark Localization with End to End Object Detection
• Aggregated Residual Transformations for Deep Neural Networks
• Path Aggregation Network for Instance Segmentation
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Scalable, High-Quality Object Detection
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Waste detection in Pomerania: non-profit project for detecting waste in environment
• A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images
• Deep Learning for Generic Object Detection: A Survey
• Active Fire Detection in Landsat-8 Imagery: a Large-Scale Dataset and a Deep-Learning Study
• Gland Instance Segmentation Using Deep Multichannel Neural Networks
• Hybrid Task Cascade for Instance Segmentation
• DeeperLab: Single-Shot Image Parser
• STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos
• A review on deep learning techniques for 3D sensed data classification
• RepPoints: Point Set Representation for Object Detection
• Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks
• Proposal-free Network for Instance-level Object Segmentation
• Learning to Fuse Things and Stuff
• Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
• Object Detection with Deep Learning: A Review
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Learning Physical Intuition of Block Towers by Example
• Learning Video Object Segmentation from Static Images
• One-Shot Instance Segmentation
• Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks
• Learning to Refine Object Segments
• 3D-BEVIS: Bird's-Eye-View Instance Segmentation
• Acquisition of Localization Confidence for Accurate Object Detection
• Deep learning for clustering of continuous gravitational wave candidates
• BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
• A Comprehensive Review of Modern Object Segmentation Approaches
• Object Discovery with a Copy-Pasting GAN
• Annotating Object Instances with a Polygon-RNN
• SOLO: Segmenting Objects by Locations
• TensorMask: A Foundation for Dense Object Segmentation
• Deep Watershed Transform for Instance Segmentation
• EmbedMask: Embedding Coupling for One-stage Instance Segmentation
• Vortex Pooling: Improving Context Representation in Semantic Segmentation
• Associatively Segmenting Instances and Semantics in Point Clouds
• Where are the Masks: Instance Segmentation with Image-level Supervision
• Panoptic Instance Segmentation on Pigs
• Simple Does It: Weakly Supervised Instance and Semantic Segmentation
• IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
• DeepBox: Learning Objectness with Convolutional Networks
• Using Deep Learning for Segmentation and Counting within Microscopy Data
• InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting
• Robust Classification with Convolutional Prototype Learning
• Instance-level Human Parsing via Part Grouping Network
• Recent Advances in Deep Learning for Object Detection
• Automatic Borescope Damage Assessments for Gas Turbine Blades via Deep Learning
• Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
• ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors
• A survey of Object Classification and Detection based on 2D/3D data
• Identifying Unknown Instances for Autonomous Driving
• Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
• Learning to Segment Every Thing
• Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation
• Deep unsupervised learning through spatial contrasting
• Deep Extreme Cut: From Extreme Points to Object Segmentation
• Instance Segmentation with Point Supervision
• Pose2Seg: Detection Free Human Instance Segmentation
• ScaleNet: Guiding Object Proposal Generation in Supermarkets and Beyond
• PolyTransform: Deep Polygon Transformer for Instance Segmentation
• Deconvolutional Feature Stacking for Weakly-Supervised Semantic Segmentation
• Weakly-supervised Salient Instance Detection
• Boundary-preserving Mask R-CNN
• LCNN: Lookup-based Convolutional Neural Network
• Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
• Actor-Action Semantic Segmentation with Region Masks
• Learning Instance Occlusion for Panoptic Segmentation
• Evaluating Generalization Ability of Convolutional Neural Networks and Capsule Networks for Image Classification via Top-2 Classification
• One-shot Texture Segmentation
• Reversible Recursive Instance-level Object Segmentation
• Prediction-Tracking-Segmentation
• Single Pixel Reconstruction for One-stage Instance Segmentation
• Learning Visual Features from Large Weakly Supervised Data
• Unseen Object Instance Segmentation for Robotic Environments
• State-Aware Tracker for Real-Time Video Object Segmentation
• Boundary-aware Instance Segmentation
• Semantic Amodal Segmentation
• Semi-convolutional Operators for Instance Segmentation
• A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection
• Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images
• SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation
• Zoom Out-and-In Network with Map Attention Decision for Region Proposal and Object Detection
• Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data
• Tag Prediction at Flickr: a View from the Darkroom
• Amodal Segmentation Based on Visible Region Segmentation and Shape Prior
• U-Net Based Multi-instance Video Object Segmentation
• Instance Shadow Detection
• Straight to Shapes: Real-time Detection of Encoded Shapes
• Seesaw Loss for Long-Tailed Instance Segmentation
• Locally Adaptive Learning Loss for Semantic Image Segmentation
• Pseudo Mask Augmented Object Detection
• Unsupervised Deep Feature Transfer for Low Resolution Image Classification
• Learning from Web Data: the Benefit of Unsupervised Object Localization
• SeGAN: Segmenting and Generating the Invisible
• Resampling Forgery Detection Using Deep Learning and A-Contrario Analysis
• 3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection
• Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks
• A^2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation
• Fast Object Segmentation Learning with Kernel-based Methods for Robotics
• Training Generative Adversarial Networks in One Stage
• RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks
• A Systematic Comparison of Deep Learning Architectures in an Autonomous Vehicle
• Pixelwise Instance Segmentation with a Dynamically Instantiated Network
• Object Detection with Mask-based Feature Encoding
• Object as Distribution
• S4Net: Single Stage Salient-Instance Segmentation
• Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation
• What leads to generalization of object proposals?
• Delving Deep into Liver Focal Lesion Detection: A Preliminary Study
• Visual Relationship Prediction via Label Clustering and Incorporation of Depth Information
• Tube-CNN: Modeling temporal evolution of appearance for object detection in video
• Phase Collaborative Network for Two-Phase Medical Image Segmentation
• Learning Visual Affordances with Target-Orientated Deep Q-Network to Grasp Objects by Harnessing Environmental Fixtures
• Generating Self-Guided Dense Annotations for Weakly Supervised Semantic Segmentation
• Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds
• Spatial contrasting for deep unsupervised learning
• Two stages for visual object tracking
• Open-World Entity Segmentation
• Feature Extraction and Classification from Planetary Science Datasets enabled by Machine Learning
• Task-driven Semantic Coding via Reinforcement Learning
• SOLO: A Simple Framework for Instance Segmentation
• S3-Net: A Fast and Lightweight Video Scene Understanding Network by Single-shot Segmentation
• Self-Reorganizing and Rejuvenating CNNs for Increasing Model Capacity Utilization
• FAIRS -- Soft Focus Generator and Attention for Robust Object Segmentation from Extreme Points
• Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation
• Quantifying the presence of graffiti in urban environments
• Object-Oriented Dynamics Learning through Multi-Level Abstraction
• Synthesizing Photorealistic Images with Deep Generative Learning
• Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction
• Motion Prediction in Visual Object Tracking
• Meta R-CNN : Towards General Solver for Instance-level Few-shot Learning
• Opening up Open-World Tracking
• Organ At Risk Segmentation with Multiple Modality
• Literature Review: Human Segmentation with Static Camera
• Snap Angle Prediction for 360${}^{\circ }$ Panoramas
• Intrinsic Image Transformation via Scale Space Decomposition
• Collaborative Annotation of Semantic Objects in Images with Multi-granularity Supervisions
• Harvesting Visual Objects from Internet Images via Deep Learning Based Objectness Assessment
• Toward Scale-Invariance and Position-Sensitive Region Proposal Networks
• Segmentation of Microscopy Data for finding Nuclei in Divergent Images
• A High-Performance Object Proposals based on Horizontal High Frequency Signal
• Affinity Derivation and Graph Merge for Instance Segmentation
• Interpreting Deep Neural Networks with Relative Sectional Propagation by Analyzing Comparative Gradients and Hostile Activations
• Universal Bounding Box Regression and Its Applications
• Vision-based Price Suggestion for Online Second-hand Items
• Semantically-Aware Strategies for Stereo-Visual Robotic Obstacle Avoidance
• A Distraction Score for Watermarks