Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing

Abstract

Human parsing has recently attracted a lot of research interests due to its huge application potentials. However existing datasets have limited number of images and annotations, and lack the variety of human appearances and the coverage of challenging cases in unconstrained environment. In this paper, we introduce a new benchmark "Look into Person (LIP)" that makes a significant advance in terms of scalability, diversity and difficulty, a contribution that we feel is crucial for future developments in human-centric analysis. This comprehensive dataset contains over 50,000 elaborately annotated images with 19 semantic part labels, which are captured from a wider range of viewpoints, occlusions and background complexity. Given these rich annotations we perform detailed analyses of the leading human parsing approaches, gaining insights into the success and failures of these methods. Furthermore, in contrast to the existing efforts on improving the feature discriminative capability, we solve human parsing by exploring a novel self-supervised structure-sensitive learning approach, which imposes human pose structures into parsing results without resorting to extra supervision (i.e., no need for specifically labeling human joints in model training). Our self-supervised learning framework can be injected into any advanced neural networks to help incorporate rich high-level knowledge regarding human joints from a global perspective and improve the parsing results. Extensive evaluations on our LIP and the public PASCAL-Person-Part dataset demonstrate the superiority of our method.

Accepted to appear in CVPR 2017

References in corpus (2)

• Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
• Joint Object and Part Segmentation using Deep Learned Potentials

Cited by in corpus (23)

• High-Resolution Representations for Labeling Pixels and Regions
• OCNet: Object Context Network for Scene Parsing
• Attention-Aware Compositional Network for Person Re-identification
• Improved Person Re-Identification Based on Saliency and Semantic Parsing with Deep Neural Network Models
• Multiple-Human Parsing in the Wild
• Fashion Meets Computer Vision: A Survey
• Instance-level Human Parsing via Part Grouping Network
• VITON: An Image-based Virtual Try-on Network
• Holistic, Instance-Level Human Parsing
• Toward Characteristic-Preserving Image-based Virtual Try-On Network
• Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis
• Detailed Human Avatars from Monocular Video
• Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention
• Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer
• Recapture as You Want
• Hierarchical semantic segmentation using modular convolutional neural networks
• SegNBDT: Visual Decision Rules for Segmentation
• Brand > Logo: Visual Analysis of Fashion Brands
• Knowledge Integration Networks for Action Recognition
• Perceive Where to Focus: Learning Visibility-aware Part-level Features for Partial Person Re-identification
• Fashion is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources
• Layout-Graph Reasoning for Fashion Landmark Detection
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN