Learning Dense Convolutional Embeddings for Semantic Segmentation

Abstract

This paper proposes a new deep convolutional neural network (DCNN) architecture that learns pixel embeddings, such that pairwise distances between the embeddings can be used to infer whether or not the pixels lie on the same region. That is, for any two pixels on the same object, the embeddings are trained to be similar; for any pair that straddles an object boundary, the embeddings are trained to be dissimilar. Experimental results show that when this embedding network is used in conjunction with a DCNN trained on semantic segmentation, there is a systematic improvement in per-pixel classification accuracy. Our contributions are integrated in the popular Caffe deep learning framework, and consist in straightforward modifications to convolution routines. As such, they can be exploited for any task involving convolution layers.

References in corpus (4)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Computing the Stereo Matching Cost with a Convolutional Neural Network
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Convolutional Feature Masking for Joint Object and Stuff Segmentation

Cited by in corpus (7)

• Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection
• Object Discovery with a Copy-Pasting GAN
• Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
• Semi-convolutional Operators for Instance Segmentation
• Learning and Segmenting Dense Voxel Embeddings for 3D Neuron Reconstruction
• Consensus Feature Network for Scene Parsing
• STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling