283 citations · 470 across the 84 of their papers we have counts for
92 papers · 1 filter
KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs
Aravindh Mahendran, Michael King, Matthew Koichi Grimes +12
We push the frontier of large-scale spatial intelligence in Vision-Language Models (VLMs) and introduce the first benchmark that probes geographical layout understanding from real-…
Whareformer: Learning to Track What is Where in Long Egocentric Videos
Jacob Chalk, Saptarshi Sinha, Dima Damen +2
The recently established 'Out of Sight, Not out of Mind' (OSNOM) task for egocentric videos focuses on tracking objects that are moved by the camera wearer, online, maintaining kno…
Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
Masatoshi Tateno, Alexandros Stergiou, Risa Shinoda +2
Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations. However, existing video-language models often fall into shortcuts b…
Towards in-the-wild Egocentric 3D Hand-Object Pose Estimation
Siddhant Bansal, Zhifan Zhu, Shashank Tripathi +3
Estimating accurate 3D hand-object pose from in-the-wild egocentric RGB remains challenging due to severe occlusions and ambiguous contact. Existing learning-based methods often st…
Gen2Balance: Generative Balancing for Long-Tailed Video Action Recognition
Prajwal Gatti, Simon Jenni, Fabian Caba Heilbron +1
We address the problem of training on long-tailed data for video action recognition. We propose to augment the training set using a text-to-video generative model, conditioned on d…
Improving and Evaluating Hand-Object Interaction Detection
Ahmad Darkhalil, Dima Damen, David Fouhey
Understanding hands and the objects they interact with, both directly and through tools, is a key step for tasks ranging from action perception to 3D reconstruction and robotics. O…
Beyond Caption-Based Queries for Video Moment Retrieval
David Pujol-Perich, Albert Clapés, Dima Damen +2
In this work, we investigate the degradation of existing VMR methods, particularly of DETR architectures, when trained on caption-based queries but evaluated on search queries. For…
Perception Test 2025: Challenge Summary and a Unified VQA Extension
Joseph Heyward, Nikhil Parthasarathy, Tyler Zhu +5
The Third Perception Test challenge was organised as a full-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2025. Its primary goal is to benc…
Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach
Masashi Hatano, Saptarshi Sinha, Jacob Chalk +3
Human motion generation is a challenging task that aims to create realistic motion imitating natural human behaviour. We focus on the well-studied behaviour of priming an object/lo…
The N-Body Problem: Parallel Execution from Single-Person Egocentric Video
Zhifan Zhu, Yifei Huang, Yoichi Sato +1
Humans can intuitively parallelise complex activities, but can a model predict this from observing a single person? Given one egocentric video, we introduce the N-Body Problem: pre…
Reconstructing Objects along Hand Interaction Timelines in Egocentric Video
Zhifan Zhu, Siddhant Bansal, Shashank Tripathi +1
We introduce the task of Reconstructing Objects along Hand Interaction Timelines (ROHIT). We first define the Hand Interaction Timeline (HIT) from a rigid object's perspective. In…
Unique Lives, Shared World: Learning from Single-Life Videos
Tengda Han, Sayna Ebrahimi, Dilara Gokay +8
We introduce the "single-life" learning paradigm, where we train a distinct vision model exclusively on egocentric videos captured by one individual. We leverage the multiple viewp…
Segmenting Collision Sound Sources in Egocentric Videos
Kranti Kumar Parida, Omar Emara, Hazel Doughty +1
Humans excel at multisensory perception and can often recognise object properties from the sound of their interactions. Inspired by this, we propose the novel task of Collision Sou…
Seeing without Pixels: Perception from Camera Trajectories
Zihui Xue, Kristen Grauman, Dima Damen +2
Can one perceive a video's content without seeing its pixels, just from the camera trajectory-the path it carves through space? This paper is the first to systematically investigat…
PointSt3R: Point Tracking through 3D Grounded Correspondence
Rhodri Guerrier, Adam W. Harley, Dima Damen
Recent advances in foundational 3D reconstruction models, such as DUSt3R and MASt3R, have shown great potential in 2D and 3D correspondence in static scenes. In this paper, we prop…
Leveraging Auxiliary Information in Text-to-Video Retrieval: A Review
Adriano Fragomeni, Dima Damen, Michael Wray
Text-to-Video (T2V) retrieval aims to identify the most relevant item from a gallery of videos based on a user's text query. Traditional methods rely solely on aligning video and t…
The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimation
Masashi Hatano, Zhifan Zhu, Hideo Saito +1
Forecasting hand motion and pose from an egocentric perspective is essential for understanding human intention. However, existing methods focus solely on predicting positions witho…
Learning from Streaming Video with Orthogonal Gradients
Tengda Han, Dilara Gokay, Joseph Heyward +6
We address the challenge of representation learning from a continuous stream of video as input, in a self-supervised manner. This differs from the standard approaches to video lear…
Leveraging Modality Tags for Enhanced Cross-Modal Video Retrieval
Adriano Fragomeni, Dima Damen, Michael Wray
Video retrieval requires aligning visual content with corresponding natural language descriptions. In this paper, we introduce Modality Auxiliary Concepts for Video Retrieval (MAC-…
HD-EPIC: A Highly-Detailed Egocentric Video Dataset
Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha +16
We present a validation dataset of newly-collected kitchen-based egocentric videos, manually annotated with highly detailed and interconnected ground-truth labels covering: recipe…
Moment of Untruth: Dealing with Negative Queries in Video Moment Retrieval
Kevin Flanagan, Dima Damen, Michael Wray
Video Moment Retrieval is a common task to evaluate the performance of visual-language models - it involves localising start and end times of moments in videos from query sentences…
EgoPoints: Advancing Point Tracking for Egocentric Videos
Ahmad Darkhalil, Rhodri Guerrier, Adam W. Harley +1
We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS eval…
Scaling 4D Representations
João Carreira, Dilara Gokay, Michael King +32
Scaling has not yet been convincingly demonstrated for pure self-supervised learning from video. However, prior work has focused evaluations on semantic-related tasks $\unicode{x20…
ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructions
Tomáš Souček, Prajwal Gatti, Michael Wray +3
The goal of this work is to generate step-by-step visual instructions in the form of a sequence of images, given an input image that provides the scene context and the sequence of…