Next-Active-Object prediction from Egocentric Videos

Abstract

Although First Person Vision systems can sense the environment from the user's perspective, they are generally unable to predict his intentions and goals. Since human activities can be decomposed in terms of atomic actions and interactions with objects, intelligent wearable systems would benefit from the ability to anticipate user-object interactions. Even if this task is not trivial, the First Person Vision paradigm can provide important cues to address this challenge. We propose to exploit the dynamics of the scene to recognize next-active-objects before an object interaction begins. We train a classifier to discriminate trajectories leading to an object activation from all others and forecast next-active-objects by analyzing fixed-length trajectory segments within a temporal sliding window. The proposed method compares favorably with respect to several baselines on the Activity of Daily Living (ADL) egocentric dataset comprising 10 hours of videos acquired by 20 subjects while performing unconstrained interactions with several objects.

References in corpus (5)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Simple Online and Realtime Tracking
• Predicting Important Objects for Egocentric Video Summarization
• Compact CNN for Indexing Egocentric Videos
• First Person Action-Object Detection with EgoNet

Cited by in corpus (17)

• Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video
• Learning to Anticipate Egocentric Actions by Imagination
• Forecasting Action through Contact Representations from First Person Video
• Self-Regulated Learning for Egocentric Video Activity Anticipation
• Visual Object Tracking in First Person Vision
• Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video
• Deep execution monitor for robot assistive tasks
• iCub Detecting Gazed Objects: A Pipeline Estimating Human Attention
• Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos
• Generative Hybrid Representations for Activity Forecasting with No-Regret Learning
• Future Person Localization in First-Person Videos
• Streaming egocentric action anticipation: An evaluation scheme and approach
• Shaping embodied agent behavior with activity-context priors from egocentric video
• Is First Person Vision Challenging for Object Tracking?
• A Self Validation Network for Object-Level Human Attention Estimation
• Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives
• SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videos