Cross-Domain Complementary Learning Using Pose for Multi-Person Part Segmentation

Abstract

Supervised deep learning with pixel-wise training labels has great successes on multi-person part segmentation. However, data labeling at pixel-level is very expensive. To solve the problem, people have been exploring to use synthetic data to avoid the data labeling. Although it is easy to generate labels for synthetic data, the results are much worse compared to those using real data and manual labeling. The degradation of the performance is mainly due to the domain gap, i.e., the discrepancy of the pixel value statistics between real and synthetic data. In this paper, we observe that real and synthetic humans both have a skeleton (pose) representation. We found that the skeletons can effectively bridge the synthetic and real domains during the training. Our proposed approach takes advantage of the rich and realistic variations of the real data and the easily obtainable labels of the synthetic data to learn multi-person part segmentation on real images without any human-annotated labels. Through experiments, we show that without any human labeling, our method performs comparably to several state-of-the-art approaches which require human labeling on Pascal-Person-Parts and COCO-DensePose datasets. On the other hand, if part labels are also available in the real-images during training, our method outperforms the supervised state-of-the-art methods by a large margin. We further demonstrate the generalizability of our method on predicting novel keypoints in real images where no real data labels are available for the novel keypoints detection. Code and pre-trained models are available at https://github.com/kevinlin311tw/CDCL-human-part-segmentation

To appear in IEEE Transactions on Circuits and Systems for Video Technology; Presented at ICCV 2019 Demonstration

References in corpus (12)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Unsupervised Domain Adaptation by Backpropagation
• Conditional Random Fields as Recurrent Neural Networks
• An Overview of Multi-Task Learning in Deep Neural Networks
• ShapeNet: An Information-Rich 3D Model Repository
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Learning from Synthetic Humans
• A Survey on Multi-Task Learning
• Domain Separation Networks
• R-CNNs for Pose Estimation and Action Detection
• Multi-source Domain Adaptation for Semantic Segmentation
• Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network

Cited by in corpus (9)

• Parameter-Efficient Person Re-identification in the 3D Space
• Gaussian Dynamic Convolution for Efficient Single-Image Segmentation
• Video Semantic Segmentation with Distortion-Aware Feature Correction
• KTN: Knowledge Transfer Network for Learning Multi-person 2D-3D Correspondences
• Deep Common Feature Mining for Efficient Video Semantic Segmentation
• To Make Yourself Invisible with Adversarial Semantic Contours
• Occlusion-Invariant Rotation-Equivariant Semi-Supervised Depth Based Cross-View Gait Pose Estimation
• Self-Learning with Rectification Strategy for Human Parsing
• Adversarial Semantic Contour for Object Detection