Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Abstract

We propose a novel weakly-supervised semantic segmentation algorithm based on Deep Convolutional Neural Network (DCNN). Contrary to existing weakly-supervised approaches, our algorithm exploits auxiliary segmentation annotations available for different categories to guide segmentations on images with only image-level class labels. To make the segmentation knowledge transferrable across categories, we design a decoupled encoder-decoder architecture with attention model. In this architecture, the model generates spatial highlights of each category presented in an image using an attention model, and subsequently generates foreground segmentation for each highlighted region using decoder. Combining attention model, we show that the decoder trained with segmentation annotations in different categories can boost the performance of weakly-supervised semantic segmentation. The proposed algorithm demonstrates substantially improved performance compared to the state-of-the-art weakly-supervised techniques in challenging PASCAL VOC 2012 dataset when our model is trained with the annotations in 60 exclusive categories in Microsoft COCO dataset.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Unsupervised Domain Adaptation by Backpropagation
• Recurrent Models of Visual Attention
• Fully Convolutional Networks for Semantic Segmentation
• Multiple Object Recognition with Visual Attention
• Learning Deconvolution Network for Semantic Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning

Cited by in corpus (6)

• Drivers Drowsiness Detection using Condition-Adaptive Representation Learning Framework
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
• Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation
• Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution
• Deconvolutional Feature Stacking for Weakly-Supervised Semantic Segmentation
• Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation