iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection

Abstract

Recent years have witnessed rapid progress in detecting and recognizing individual object instances. To understand the situation in a scene, however, computers need to recognize how humans interact with surrounding objects. In this paper, we tackle the challenging task of detecting human-object interactions (HOI). Our core idea is that the appearance of a person or an object instance contains informative cues on which relevant parts of an image to attend to for facilitating interaction prediction. To exploit these cues, we propose an instance-centric attention module that learns to dynamically highlight regions in an image conditioned on the appearance of each instance. Such an attention-based network allows us to selectively aggregate features relevant for recognizing HOIs. We validate the efficacy of the proposed network on the Verb in COCO and HICO-DET datasets and show that our approach compares favorably with the state-of-the-arts.

BMVC 2018. Project webpage: https://gaochen315.github.io/iCAN/ Code: https://github.com/vt-vl-lab/iCAN

Cited by in corpus (26)

• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection
• Transferable Interactiveness Knowledge for Human-Object Interaction Detection
• Forecasting Action through Contact Representations from First Person Video
• HAKE: Human Activity Knowledge Engine
• Learning Human-Object Interaction Detection using Interaction Points
• PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection
• ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos
• Pose-aware Multi-level Feature Network for Human Object Interaction Detection
• HOTR: End-to-End Human-Object Interaction Detection with Transformers
• QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information
• Interact as You Intend: Intention-Driven Human-Object Interaction Detection
• VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions
• Unseen No More: Unlocking the Potential of CLIP for Generative Zero-shot HOI Detection
• Reformulating HOI Detection as Adaptive Set Prediction
• Cascaded Human-Object Interaction Recognition
• Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detection
• Affordance Transfer Learning for Human-Object Interaction Detection
• Effective Actor-centric Human-object Interaction Detection
• ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection
• GTNet:Guided Transformer Network for Detecting Human-Object Interactions
• Detecting Human-to-Human-or-Object (H2O) Interactions with DIABOLO
• Parsing R-CNN for Instance-Level Human Analysis
• Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
• RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection
• Human-Object Interaction Detection via Weak Supervision