Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing
arXiv:2103.04570
Abstract
To address the challenging task of instance-aware human part parsing, a new bottom-up regime is proposed to learn category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner. It is a compact, efficient and powerful framework that exploits structural information over different human granularities and eases the difficulty of person partitioning. Specifically, a dense-to-sparse projection field, which allows explicitly associating dense human semantics with sparse keypoints, is learnt and progressively improved over the network feature pyramid for robustness. Then, the difficult pixel grouping problem is cast as an easier, multi-person joint assembling task. By formulating joint association as maximum-weight bipartite matching, a differentiable solution is developed to exploit projected gradient descent and Dykstra's cyclic projection algorithm. This makes our method end-to-end trainable and allows back-propagating the grouping error to directly supervise multi-granularity human representation learning. This is distinguished from current bottom-up human parsers or pose estimators which require sophisticated post-processing or heuristic greedy algorithms. Experiments on three instance-aware human parsing datasets show that our model outperforms other bottom-up alternatives with much more efficient inference.
CVPR 2021 (Oral). Code: https://github.com/tfzhou/MG-HumanParsing
References in corpus (14)
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Associative Embedding: End-to-End Learning for Joint Detection and Grouping
- Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
- RepPoints: Point Set Representation for Object Detection
- Instance-aware Semantic Segmentation via Multi-task Network Cascades
- Towards Accurate Multi-person Pose Estimation in the Wild
- Devil in the Details: Towards Accurate Single and Multiple Human Parsing
- Multiple-Human Parsing in the Wild
- CARAFE: Content-Aware ReAssembly of FEatures
- Instance-level Human Parsing via Part Grouping Network
- Holistic, Instance-Level Human Parsing
- Dense RepPoints: Representing Visual Objects with Dense Point Sets
- Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation
- Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation