283 citations · 774 across the 55 of their papers we have counts for
65 papers
Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark
Maria Santos-Villafranca, Jesus Bermudez-cameo, Alejandro Perez-Yus +2
To operate in the physical world, embodied agents must perceive their environment in an "always-on" fashion, selectively accessing the most informative sensors to balance energy co…
EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation
Rosario Leonardi, Francesco Ragusa, Daniele Materia +4
Collecting large-scale egocentric video datasets with dense spatial and temporal annotations is costly, slow, and often constrained by environmental biases, privacy constraints, an…
Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videos
Daniele Materia, Francesco Ragusa, Giovanni Maria Farinella
The ability to anticipate human-object interactions is highly desirable in an intelligent assistive system in order to guide users during daily life activities and understand their…
Leveraging Synthetic Data for Enhancing Egocentric Hand-Object Interaction Detection
Rosario Leonardi, Antonino Furnari, Francesco Ragusa +1
In this work, we explore the role of synthetic data in improving the detection of Hand-Object Interactions from egocentric images. Through extensive experimentation and comparative…
ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios
Francesco Ragusa, Rosario Leonardi, Michele Mazzamuto +6
Understanding human behavior from complementary egocentric (ego) and exocentric (exo) points of view enables the development of systems that can support workers in industrial envir…
Integrating Affordances and Attention models for Short-Term Object Interaction Anticipation
Lorenzo Mur Labadia, Ruben Martinez-Cantin, Jose J. Guerrero +2
Short Term object-interaction Anticipation consists in detecting the location of the next active objects, the noun and verb categories of the interaction, as well as the time to co…
ProSkill: Segment-Level Skill Assessment in Procedural Videos
Michele Mazzamuto, Daniele Di Mauro, Gianpiero Francesca +2
Skill assessment in procedural videos is crucial for the objective evaluation of human performance in settings such as manufacturing and procedural daily tasks. Current research on…
GlovEgo-HOI: Bridging the Synthetic-to-Real Gap for Industrial Egocentric Human-Object Interaction Detection
Alfio Spoto, Rosario Leonardi, Francesco Ragusa +1
Egocentric Human-Object Interaction (EHOI) analysis is crucial for industrial safety, yet the development of robust models is hindered by the scarcity of annotated domain-specific…
SignIT: A Comprehensive Dataset and Multimodal Analysis for Italian Sign Language Recognition
Alessia Micieli, Giovanni Maria Farinella, Francesco Ragusa
In this work we present SignIT, a new dataset to study the task of Italian Sign Language (LIS) recognition. The dataset is composed of 644 videos covering 3.33 hours. We manually a…
Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistance
Francesco Ragusa, Michele Mazzamuto, Rosario Forte +5
We present Ego-EXTRA, a video-language Egocentric Dataset for EXpert-TRAinee assistance. Ego-EXTRA features 50 hours of unscripted egocentric videos of subjects performing procedur…
Learning Egocentric In-Hand Object Segmentation through Weak Supervision from Human Narrations
Nicola Messina, Rosario Leonardi, Luca Ciampi +4
Pixel-level recognition of objects manipulated by the user from egocentric images enables key applications spanning assistive technologies, industrial safety, and activity monitori…
Mamba-OTR: a Mamba-based Solution for Online Take and Release Detection from Untrimmed Egocentric Video
Alessandro Sebastiano Catinello, Giovanni Maria Farinella, Antonino Furnari
This work tackles the problem of Online detection of Take and Release (OTR) of an object in untrimmed egocentric videos. This task is challenging due to severe label imbalance, wit…
Efficient Calisthenics Skills Classification through Foreground Instance Selection and Depth Estimation
Antonio Finocchiaro, Giovanni Maria Farinella, Antonino Furnari
Calisthenics skill classification is the computer vision task of inferring the skill performed by an athlete from images, enabling automatic performance assessment and personalized…
Calisthenics Skills Temporal Video Segmentation
Antonio Finocchiaro, Giovanni Maria Farinella, Antonino Furnari
Calisthenics is a fast-growing bodyweight discipline that consists of different categories, one of which is focused on skills. Skills in calisthenics encompass both static and dyna…
A Real-Time System for Egocentric Hand-Object Interaction Detection in Industrial Domains
Antonio Finocchiaro, Alessandro Sebastiano Catinello, Michele Mazzamuto +3
Hand-object interaction detection remains an open challenge in real-time applications, where intuitive user experiences depend on fast and accurate detection of interactions with s…
How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?
Giuseppe Lando, Rosario Forte, Giovanni Maria Farinella +1
We investigate whether off-the-shelf Multimodal Large Language Models (MLLMs) can tackle Online Episodic-Memory Video Question Answering (OEM-VQA) without additional training. Our…
EASG-Bench: Video Q&A Benchmark with Egocentric Action Scene Graphs
Ivan Rodin, Tz-Ying Wu, Kyle Min +4
We introduce EASG-Bench, a question-answering benchmark for egocentric videos where the question-answering pairs are created from spatio-temporally grounded dynamic scene graphs ca…
Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videos
Luigi Seminara, Giovanni Maria Farinella, Antonino Furnari
We introduce a gradient-based approach for learning task graphs from procedural activities, improving over hand-crafted methods. Our method directly optimizes edge weights via maxi…
Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memory
Zaira Manigrasso, Matteo Dunnhofer, Antonino Furnari +6
Episodic memory retrieval enables wearable cameras to recall objects or events previously observed in video. However, existing formulations assume an "offline" setting with full vi…
Semantically Guided Action Anticipation
Anxhelo Diko, Antonino Furnari, Luigi Cinque +1
Unsupervised domain adaptation remains a critical challenge in enabling the knowledge transfer of models across unseen domains. Existing methods struggle to balance the need for do…
TI-PREGO: Chain of Thought and In-Context Learning for Online Mistake Detection in PRocedural EGOcentric Videos
Leonardo Plini, Luca Scofano, Edoardo De Matteis +6
Identifying procedural errors online from egocentric videos is a critical yet challenging task across various domains, including manufacturing, healthcare, and skill-based training…
ZARRIO @ Ego4D Short Term Object Interaction Anticipation Challenge: Leveraging Affordances and Attention-based models for STA
Lorenzo Mur-Labadia, Ruben Martinez-Cantin, Josechu Guerrero-Campo +1
Short-Term object-interaction Anticipation (STA) consists of detecting the location of the next-active objects, the noun and verb categories of the interaction, and the time to con…
AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipation
Lorenzo Mur-Labadia, Ruben Martinez-Cantin, Josechu Guerrero +2
Short-Term object-interaction Anticipation consists of detecting the location of the next-active objects, the noun and verb categories of the interaction, and the time to contact f…
Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activities
Michele Mazzamuto, Antonino Furnari, Yoichi Sato +1
We address the challenge of unsupervised mistake detection in egocentric video of skilled human activities through the analysis of gaze signals. While traditional methods rely on m…