283 citations · 682 across the 52 of their papers we have counts for
13 papers · 1 filter
Retrieval-Augmented Online Textual Memory for Episodic Memory Video Question Answering
Raffaele Cali, Giuseppe Lando, Rosario Forte +1
Long egocentric videos pose a central challenge for multimodal language models: the visual stream must be compressed into a memory representation that can be searched efficiently w…
GrammarRL: Effective Grammar-Constrained Decoding via Reinforcement Learning
Gabriele Tuccio, Antonino Furnari, Aldo Gangemi +1
Grammar-constrained generation guarantees syntactic validity, but can substantially degrade semantic quality when the model's preferred outputs are poorly aligned with the imposed…
EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos
Shulin Tian, Junsu Kim, Shuai Liu +17
Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolving object and task states. Too…
SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translation
Zeno Testa, Antonino Furnari, Lorenzo Baraldi +1
Sign Language Translation (SLT) is typically evaluated with surface-form metrics such as BLEU and ROUGE, which reward lexical overlap but do not directly measure whether a translat…
EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
Rosario Forte, Giuseppe Lando, Antonino Furnari
Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools fo…
Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark
Maria Santos-Villafranca, Jesus Bermudez-cameo, Alejandro Perez-Yus +2
To operate in the physical world, embodied agents must perceive their environment in an "always-on" fashion, selectively accessing the most informative sensors to balance energy co…
RECIPE: Procedural Planning via Grounding in Instructional Video
Luigi Seminara, Antonino Furnari, Lorenzo Torresani
Visual planning asks a model to generate the remaining steps of a procedure in natural language given a partial video context and a goal. Progress on this task is bottlenecked by a…
Leveraging Synthetic Data for Enhancing Egocentric Hand-Object Interaction Detection
Rosario Leonardi, Antonino Furnari, Francesco Ragusa +1
In this work, we explore the role of synthetic data in improving the detection of Hand-Object Interactions from egocentric images. Through extensive experimentation and comparative…
ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios
Francesco Ragusa, Rosario Leonardi, Michele Mazzamuto +6
Understanding human behavior from complementary egocentric (ego) and exocentric (exo) points of view enables the development of systems that can support workers in industrial envir…
ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning in Instructional Videos
Luigi Seminara, Davide Moltisanti, Antonino Furnari
Procedural planning aims to predict a sequence of actions that transforms an initial visual state into a desired goal, a fundamental ability for intelligent agents operating in com…
Integrating Affordances and Attention models for Short-Term Object Interaction Anticipation
Lorenzo Mur Labadia, Ruben Martinez-Cantin, Jose J. Guerrero +2
Short Term object-interaction Anticipation consists in detecting the location of the next active objects, the noun and verb categories of the interaction, as well as the time to co…
Exploring Multimodal LMMs for Online Episodic Memory Question Answering on the Edge
Giuseppe Lando, Rosario Forte, Antonino Furnari
We investigate the feasibility of using Multimodal Large Language Models (MLLMs) for real-time online episodic memory question answering. While cloud offloading is common, it raise…
ProSkill: Segment-Level Skill Assessment in Procedural Videos
Michele Mazzamuto, Daniele Di Mauro, Gianpiero Francesca +2
Skill assessment in procedural videos is crucial for the objective evaluation of human performance in settings such as manufacturing and procedural daily tasks. Current research on…