Learning Compositional Neural Information Fusion for Human Parsing
arXiv:2001.06804
Abstract
This work proposes to combine neural networks with the compositional hierarchy of human bodies for efficient and complete human parsing. We formulate the approach as a neural information fusion framework. Our model assembles the information from three inference processes over the hierarchy: direct inference (directly predicting each part of a human body using image information), bottom-up inference (assembling knowledge from constituent parts), and top-down inference (leveraging context from parent nodes). The bottom-up and top-down inferences explicitly model the compositional and decompositional relations in human bodies, respectively. In addition, the fusion of multi-source information is conditioned on the inputs, i.e., by estimating and considering the confidence of the sources. The whole model is end-to-end differentiable, explicitly modeling information flows and structures. Our approach is extensively evaluated on four popular datasets, outperforming the state-of-the-arts in all cases, with a fast processing speed of 23fps. Our code and results have been released to help ease future research in this direction.
ICCV2019. Websie: https://github.com/ZzzjzzZ/CompositionalHumanParsing
References in corpus (7)
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Deep Human Parsing with Active Template Regression
- Be Your Own Prada: Fashion Synthesis with Structural Coherence
- Joint Object and Part Segmentation using Deep Learned Potentials
- Instance-level Human Parsing via Part Grouping Network
- Holistic, Instance-Level Human Parsing