22 citations · 76 across the 17 of their papers we have counts for
20 papers
SignMatch: Matching Dictionary Signs to Continuous Sign Language Video
Ryan Wong, Youngjoon Jang, Liliane Momeni +2
The objective of this paper is to match dictionary sign videos to corresponding signs in continuous signing videos, where a match is defined by the visual similarity alone - the ha…
Efficiently Reconstructing Dynamic Scenes One D4RT at a Time
Chuhan Zhang, Guillaume Le Moing, Skanda Koppula +11
Understanding and reconstructing the complex geometry and motion of dynamic scenes from video remains a formidable challenge in computer vision. This paper introduces D4RT, a simpl…
Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment
Youngjoon Jang, Liliane Momeni, Zifan Jiang +3
Our aim is to develop a unified model for sign language understanding, that performs sign language translation (SLT) and sign-subtitle alignment (SSA). Together, these two tasks en…
Segment, Embed, and Align: A Universal Recipe for Aligning Subtitles to Signing
Zifan Jiang, Youngjoon Jang, Liliane Momeni +3
The goal of this work is to develop a universal approach for aligning subtitles (i.e., spoken language text with corresponding timestamps) to continuous sign language videos. Prior…
SAM 3: Segment Anything with Concepts
Nicolas Carion, Laura Gustafson, Yuan-Ting Hu +35
We present Segment Anything Model (SAM) 3, a unified model that detects, segments, and tracks objects in images and videos based on concept prompts, which we define as either short…
SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications
Yana Hasson, Pauline Luc, Liliane Momeni +10
In recent years, there has been a proliferation of spatiotemporal foundation models in different scientific disciplines. While promising, these models are often domain-specific and…
Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues
Youngjoon Jang, Haran Raajesh, Liliane Momeni +2
Our objective is to translate continuous sign language into spoken language text. Inspired by the way human interpreters rely on context for accurate translation, we incorporate ad…
A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervision
Charles Raude, K R Prajwal, Liliane Momeni +4
In this work, our goals are two fold: large-vocabulary continuous sign language recognition (CSLR), and sign language retrieval. To this end, we introduce a multi-task Transformer…
Verbs in Action: Improving verb understanding in video-language models
Liliane Momeni, Mathilde Caron, Arsha Nagrani +2
Understanding verbs is crucial to modelling how people and objects interact with each other and the environment through space and time. Recently, state-of-the-art video-language mo…
Large Language Models are Few-shot Publication Scoopers
Samuel Albanie, Liliane Momeni, João F. Henriques
Driven by recent advances AI, we passengers are entering a golden age of scientific discovery. But golden for whom? Confronting our insecurity that others may beat us to the most a…
Weakly-supervised Fingerspelling Recognition in British Sign Language Videos
K R Prajwal, Hannah Bull, Liliane Momeni +3
The goal of this work is to detect and recognize sequences of letters signed using fingerspelling in British Sign Language (BSL). Previous fingerspelling recognition methods have n…
Automatic dense annotation of large-vocabulary sign language videos
Liliane Momeni, Hannah Bull, K R Prajwal +3
Recently, sign language researchers have turned to sign language interpreted TV broadcasts, comprising (i) a video of continuous signing and (ii) subtitles corresponding to the aud…
Scaling up sign spotting through sign language dictionaries
Gül Varol, Liliane Momeni, Samuel Albanie +2
The focus of this work is - given a video of an isolated sign, our task is to identify and it has been signed in a cont…
BBC-Oxford British Sign Language Dataset
Samuel Albanie, Gül Varol, Liliane Momeni +8
In this work, we introduce the BBC-Oxford British Sign Language (BOBSL) dataset, a large-scale video collection of British Sign Language (BSL). BOBSL is an extended and publicly re…
Visual Keyword Spotting with Attention
K R Prajwal, Liliane Momeni, Triantafyllos Afouras +1
In this paper, we consider the task of spotting spoken keywords in silent video sequences -- also known as visual keyword spotting. To this end, we investigate Transformer-based mo…
Aligning Subtitles in Sign Language Videos
Hannah Bull, Triantafyllos Afouras, Gül Varol +3
The goal of this work is to temporally align asynchronous subtitles in sign language videos. In particular, we focus on sign-language interpreted TV broadcast data comprising (i) a…
Read and Attend: Temporal Localisation in Sign Language Videos
Gül Varol, Liliane Momeni, Samuel Albanie +2
The objective of this work is to annotate sign instances across a broad vocabulary in continuous sign language. We train a Transformer model to ingest a continuous signing stream a…
Watch, read and lookup: learning to spot signs from multiple supervisors
Liliane Momeni, Gül Varol, Samuel Albanie +2
The focus of this work is sign spotting - given a video of an isolated sign, our task is to identify whether and where it has been signed in a continuous, co-articulated sign langu…
Seeing wake words: Audio-visual Keyword Spotting
Liliane Momeni, Triantafyllos Afouras, Themos Stafylakis +2
The goal of this work is to automatically determine whether and when a word of interest is spoken by a talking face, with or without the audio. We propose a zero-shot method suitab…
BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues
Samuel Albanie, Gül Varol, Liliane Momeni +4
Recent progress in fine-grained gesture and action classification, and machine translation, point to the possibility of automated sign language recognition becoming a reality. A ke…