Semantic Instance Segmentation with a Discriminative Loss Function

Abstract

Semantic instance segmentation remains a challenging task. In this work we propose to tackle the problem with a discriminative loss function, operating at the pixel level, that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step. The loss function encourages the network to map each pixel to a point in feature space so that pixels belonging to the same instance lie close together while different instances are separated by a wide margin. Our approach of combining an off-the-shelf network with a principled loss function inspired by a metric learning objective is conceptually simple and distinct from recent efforts in instance segmentation. In contrast to previous works, our method does not rely on object proposals or recurrent mechanisms. A key contribution of our work is to demonstrate that such a simple setup without bells and whistles is effective and can perform on par with more complex methods. Moreover, we show that it does not suffer from some of the limitations of the popular detect-and-segment approaches. We achieve competitive performance on the Cityscapes and CVPPP leaf segmentation benchmarks.

Published at "Deep Learning for Robotic Vision", workshop at CVPR 2017

References in corpus (16)

• Multi-Scale Context Aggregation by Dilated Convolutions
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Simultaneous Detection and Segmentation
• Gland Instance Segmentation Using Deep Multichannel Neural Networks
• Proposal-free Network for Instance-level Object Segmentation
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Deep Watershed Transform for Instance Segmentation
• Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• End-to-end people detection in crowded scenes
• InstanceCut: from Edges to Instances with MultiCut
• Learning to decompose for object detection and instance segmentation
• Boundary-aware Instance Segmentation

Cited by in corpus (113)

• YOLACT++: Better Real-time Instance Segmentation
• SOLOv2: Dynamic and Fast Instance Segmentation
• CCNet: Criss-Cross Attention for Semantic Segmentation
• DeeperLab: Single-Shot Image Parser
• STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos
• From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
• Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey
• PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain
• Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images
• Recurrent Neural Networks for Semantic Instance Segmentation
• Position-Aware Relation Learning for RGB-Thermal Salient Object Detection
• Key Points Estimation and Point Instance Segmentation Approach for Lane Detection
• 3D-BEVIS: Bird's-Eye-View Instance Segmentation
• Multiple-Human Parsing in the Wild
• SOLQ: Segmenting Objects by Learning Queries
• ISTR: End-to-End Instance Segmentation with Transformers
• FPCC: Fast Point Cloud Clustering based Instance Segmentation for Industrial Bin-picking
• BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
• Deep Learning for 3D Point Clouds: A Survey
• Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
• Object Discovery with a Copy-Pasting GAN
• Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
• SOLO: Segmenting Objects by Locations
• Learning Gaussian Instance Segmentation in Point Clouds
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
• Associatively Segmenting Instances and Semantics in Point Clouds
• EmbedMask: Embedding Coupling for One-stage Instance Segmentation
• Panoptic Instance Segmentation on Pigs
• 3D-LaneNet+: Anchor Free Lane Detection using a Semi-Local Representation
• SSAP: Single-Shot Instance Segmentation With Affinity Pyramid
• Instance Segmentation by Deep Coloring
• 3D Instance Segmentation of MVS Buildings
• Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation
• An End-to-End Network for Panoptic Segmentation
• ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors
• TextMountain: Accurate Scene Text Detection via Instance Segmentation
• Learning a Spatio-Temporal Embedding for Video Instance Segmentation
• Learning Category- and Instance-Aware Pixel Embedding for Fast Panoptic Segmentation
• Identifying Unknown Instances for Autonomous Driving
• RDSNet: A New Deep Architecture for Reciprocal Object Detection and Instance Segmentation
• TTPLA: An Aerial-Image Dataset for Detection and Segmentation of Transmission Towers and Power Lines
• CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution
• GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement
• Instance Embedding Transfer to Unsupervised Video Object Segmentation
• Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning
• Towards Lightweight Lane Detection by Optimizing Spatial Embedding
• Boundary-preserving Mask R-CNN
• JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds
• Beef Cattle Instance Segmentation Using Fully Convolutional Neural Network
• Point Cloud Instance Segmentation using Probabilistic Embeddings
• Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
• Multi-scale attention-based instance segmentation for measuring crystals with large size variation
• Unseen Object Instance Segmentation for Robotic Environments
• PatchPerPix for Instance Segmentation
• PanoNet: Real-time Panoptic Segmentation through Position-Sensitive Feature Embedding
• Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks
• Embedding-based Instance Segmentation in Microscopy
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution
• Semi-convolutional Operators for Instance Segmentation
• Learning to embed semantic similarity for joint image-text retrieval
• Improving Pixel Embedding Learning through Intermediate Distance Regression Supervision for Instance Segmentation
• BoundarySqueeze: Image Segmentation as Boundary Squeezing
• Object Discovery in Videos as Foreground Motion Clustering
• End-to-End Human Instance Matting
• Learning and Segmenting Dense Voxel Embeddings for 3D Neuron Reconstruction
• Semi-Local 3D Lane Detection and Uncertainty Estimation
• OccuSeg: Occupancy-aware 3D Instance Segmentation
• Geometry-Aware Instance Segmentation with Disparity Maps
• 3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection
• A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
• RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks
• Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds
• iShape: A First Step Towards Irregular Shape Instance Segmentation
• GeoLayout: Geometry Driven Room Layout Estimation Based on Depth Maps of Planes
• A^2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation
• Distance to Center of Mass Encoding for Instance Segmentation
• Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans
• CASNet: Common Attribute Support Network for image instance and panoptic segmentation
• Real-Time Panoptic Segmentation from Dense Detections
• Neural Mesh Refiner for 6-DoF Pose Estimation
• SOLO: A Simple Framework for Instance Segmentation
• supervised adptive threshold network for instance segmentation
• LeafMask: Towards Greater Accuracy on Leaf Segmentation
• Transformer Assisted Convolutional Network for Cell Instance Segmentation
• Attend to Who You Are: Supervising Self-Attention for Keypoint Detection and Instance-Aware Association
• Dynamic Convolution for 3D Point Cloud Instance Segmentation
• Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation
• Semantic Segmentation and Object Detection Towards Instance Segmentation: Breast Tumor Identification
• Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds
• Reinforced Coloring for End-to-End Instance Segmentation
• Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation
• SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor
• Flow Based Self-supervised Pixel Embedding for Image Segmentation
• Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding
• FoxNet: A Multi-face Alignment Method
• Instance segmentation with the number of clusters incorporated in embedding learning
• Bounding Box Embedding for Single Shot Person Instance Segmentation
• A Coarse-to-Fine Instance Segmentation Network with Learning Boundary Representation
• MaskPlus: Improving Mask Generation for Instance Segmentation
• Layered Embeddings for Amodal Instance Segmentation
• VideoClick: Video Object Segmentation with a Single Click
• Stateless actor-critic for instance segmentation with high-level priors
• Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images
• Spatial Sampling Network for Fast Scene Understanding
• A Dataset for Lane Instance Segmentation in Urban Environments
• Indoor Panorama Planar 3D Reconstruction via Divide and Conquer
• Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy
• Instance Segmentation of Biomedical Images with an Object-aware Embedding Learned with Local Constraints
• MNEW: Multi-domain Neighborhood Embedding and Weighting for Sparse Point Clouds Segmentation
• Pixel Consensus Voting for Panoptic Segmentation
• Region segmentation via deep learning and convex optimization
• Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision