Cascaded Human-Object Interaction Recognition

Abstract

Rapid progress has been witnessed for human-object interaction (HOI) recognition, but most existing models are confined to single-stage reasoning pipelines. Considering the intrinsic complexity of the task, we introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. Each of the two networks is also connected to its predecessor at the previous stage, enabling cross-stage information propagation. The interaction recognition network has two crucial parts: a relation ranking module for high-quality HOI proposal selection and a triple-stream classifier for relation prediction. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding. Further beyond relation detection on a bounding-box level, we make our framework flexible to perform fine-grained pixel-wise relation segmentation; this provides a new glimpse into better relation modeling. Our approach reached the $1^{st}$ place in the ICCV2019 Person in Context Challenge, on both relation detection and segmentation tasks. It also shows promising results on V-COCO.

arXiv commentAccepted to CVPR 2020. Winner of the ICCV-2019 PIC Challenge on both HOIW and PIC tracks. Code: https://github.com/tfzhou/C-HOI

References in corpus (10)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• RetinaFace: Single-stage Dense Face Localisation in the Wild
• Visual Semantic Role Labeling
• iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection
• Hybrid Task Cascade for Instance Segmentation
• Learning Conditioned Graph Structures for Interpretable Visual Question Answering
• Cascade R-CNN: High Quality Object Detection and Instance Segmentation
• Learning Human-Object Interactions by Graph Parsing Neural Networks
• Reasoning Visual Dialogs with Structural and Partial Observations
• Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning

Cited by in corpus (1)

• Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detection