Joint Object and Part Segmentation using Deep Learned Potentials

Abstract

Segmenting semantic objects from images and parsing them into their respective semantic parts are fundamental steps towards detailed object understanding in computer vision. In this paper, we propose a joint solution that tackles semantic object and part segmentation simultaneously, in which higher object-level context is provided to guide part segmentation, and more detailed part-level localization is utilized to refine object segmentation. Specifically, we first introduce the concept of semantic compositional parts (SCP) in which similar semantic parts are grouped and shared among different objects. A two-channel fully convolutional network (FCN) is then trained to provide the SCP and object potentials at each pixel. At the same time, a compact set of segments can also be obtained from the SCP predictions of the network. Given the potentials and the generated segments, in order to explore long-range context, we finally construct an efficient fully connected conditional random field (FCRF) to jointly predict the final object and part labels. Extensive evaluation on three different datasets shows that our approach can mutually enhance the performance of object and part segmentation, and outperforms the current state-of-the-art on both tasks.

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Fully Convolutional Networks for Semantic Segmentation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Simultaneous Detection and Segmentation
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
• Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
• Efficient piecewise training of deep structured models for semantic segmentation
• Deformable Part Models are Convolutional Neural Networks
• segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection

Cited by in corpus (15)

• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Devil in the Details: Towards Accurate Single and Multiple Human Parsing
• Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
• Semantic Object Parsing with Local-Global Long Short-Term Memory
• Holistic, Instance-Level Human Parsing
• Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net
• Pose-Guided Human Parsing with Deep Learned Features
• Joint Multi-Person Pose Estimation and Semantic Part Segmentation
• SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks
• Surveillance Video Parsing with Single Frame Supervision
• Learning Compositional Neural Information Fusion for Human Parsing
• Semantic Object Parsing with Graph LSTM
• DeepVoting: A Robust and Explainable Deep Network for Semantic Part Detection under Partial Occlusion
• End-to-end learning potentials for structured attribute prediction
• Objects as context for detecting their semantic parts