283 citations · 470 across the 84 of their papers we have counts for
13 papers · 1 filter
Perception Test 2023: A Summary of the First Challenge And Outcome
Joseph Heyward, João Carreira, Dima Damen +2
The First Perception Test challenge was held as a half-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2023, with the goal of benchmarking st…
Get a Grip: Reconstructing Hand-Object Stable Grasps in Egocentric Videos
Zhifan Zhu, Dima Damen
We propose the task of Hand-Object Stable Grasp Reconstruction (HO-SGR), the reconstruction of frames during which the hand is stably holding the object. We first develop the stabl…
GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos
Tomáš Souček, Dima Damen, Michael Wray +2
We address the task of generating temporally consistent and physically plausible images of actions and object state transformations. Given an input image and a text prompt describi…
Learning from One Continuous Video Stream
João Carreira, Michael King, Viorica Pătrăucean +9
We introduce a framework for online learning from a single continuous video stream -- the way people and animals learn, without mini-batches, data augmentation or shuffling. This p…
Centre Stage: Centricity-based Audio-Visual Temporal Action Detection
Hanyuan Wang, Majid Mirmehdi, Dima Damen +1
Previous one-stage action detection approaches have modelled temporal dependencies using only the visual modality. In this paper, we explore different strategies to incorporate the…
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani +98
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric…
Learning Temporal Sentence Grounding From Narrated EgoVideos
Kevin Flanagan, Dima Damen, Michael Wray
The onset of long-form egocentric datasets such as Ego4D and EPIC-Kitchens presents a new challenge for the task of Temporal Sentence Grounding (TSG). Compared to traditional bench…
An Outlook into the Future of Egocentric Vision
Chiara Plizzari, Gabriele Goletto, Antonino Furnari +5
What will the future be? We wonder! In this survey, we explore the gap between current research in egocentric vision and the ever-anticipated future, where wearable computing, with…
EPIC Fields: Marrying 3D Geometry and Video Understanding
Vadim Tschernezki, Ahmad Darkhalil, Zhifan Zhu +5
Neural rendering is fuelling a unification of learning, 3D geometry and video understanding that has been waiting for more than two decades. Progress, however, is still hampered by…
What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations
Chiara Plizzari, Toby Perrett, Barbara Caputo +1
We propose and address a new generalisation problem: can a model trained for action recognition successfully classify actions when they are performed within a previously unseen sce…
Perception Test: A Diagnostic Benchmark for Multimodal Video Models
Viorica Pătrăucean, Lucas Smaira, Ankush Gupta +21
We propose a novel multimodal video benchmark - the Perception Test - to evaluate the perception and reasoning skills of pre-trained multimodal models (e.g. Flamingo, SeViLA, or GP…
Use Your Head: Improving Long-Tail Video Recognition
Toby Perrett, Saptarshi Sinha, Tilo Burghardt +2
This paper presents an investigation into long-tail video recognition. We demonstrate that, unlike naturally-collected video datasets and existing long-tail image benchmarks, curre…
Epic-Sounds: A Large-scale Dataset of Actions That Sound
Jaesung Huh, Jacob Chalk, Evangelos Kazakos +2
We introduce EPIC-SOUNDS, a large-scale dataset of audio annotations capturing temporal extents and class labels within the audio stream of the egocentric videos. We propose an ann…