Instance-level Human Parsing via Part Grouping Network

Abstract

Instance-level human parsing towards real-world human analysis scenarios is still under-explored due to the absence of sufficient data resources and technical difficulty in parsing multiple instances in a single pass. Several related works all follow the "parsing-by-detection" pipeline that heavily relies on separately trained detection models to localize instances and then performs human parsing for each instance sequentially. Nonetheless, two discrepant optimization targets of detection and parsing lead to suboptimal representation learning and error accumulation for final results. In this work, we make the first attempt to explore a detection-free Part Grouping Network (PGN) for efficiently parsing multiple people in an image in a single pass. Our PGN reformulates instance-level human parsing as two twinned sub-tasks that can be jointly learned and mutually refined via a unified network: 1) semantic part segmentation for assigning each pixel as a human part (e.g., face, arms); 2) instance-aware edge detection to group semantic parts into distinct person instances. Thus the shared intermediate representation would be endowed with capabilities in both characterizing fine-grained parts and inferring instance belongings of each part. Finally, a simple instance partition process is employed to get final results during inference. We conducted experiments on PASCAL-Person-Part dataset and our PGN outperforms all state-of-the-art methods. Furthermore, we show its superiority on a newly collected multi-person parsing dataset (CIHP) including 38,280 diverse images, which is the largest dataset so far and can facilitate more advanced human analysis. The CIHP benchmark and our source code are available at http://sysu-hcp.net/lip/.

arXiv commentAccepted by ECCV 2018 (Oral)

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Richer Convolutional Features for Edge Detection
• Learning to Segment Object Candidates
• Deep Human Parsing with Active Template Regression
• Pyramid Scene Parsing Network
• Simultaneous Detection and Segmentation
• Proposal-free Network for Instance-level Object Segmentation
• Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
• Deep Watershed Transform for Instance Segmentation
• Semantic Object Parsing with Local-Global Long Short-Term Memory
• Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
• InstanceCut: from Edges to Instances with MultiCut
• Holistic, Instance-Level Human Parsing
• Reversible Recursive Instance-level Object Segmentation

Cited by in corpus (12)

• Devil in the Details: Towards Accurate Single and Multiple Human Parsing
• Fashion Meets Computer Vision: A Survey
• Coordinate-based Texture Inpainting for Pose-Guided Image Generation
• Hierarchical Human Parsing with Typed Part-Relation Reasoning
• BoundarySqueeze: Image Segmentation as Boundary Squeezing
• Learning Compositional Neural Information Fusion for Human Parsing
• Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing
• GAC-GAN: A General Method for Appearance-Controllable Human Video Motion Transfer
• Correlating Edge, Pose with Parsing
• Parsing R-CNN for Instance-Level Human Analysis
• A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos
• Quality-Aware Network for Face Parsing