80 citations · 136 across the 10 of their papers we have counts for
8 papers · 1 filter
Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Prediction
Oana Ignat, Santiago Castro, Weiji Li +1
We introduce the task of automatic human action co-occurrence identification, i.e., determine whether two human actions can co-occur in the same interval of time. We create and mak…
Scalable Performance Analysis for Vision-Language Models
Santiago Castro, Oana Ignat, Rada Mihalcea
Joint vision-language models have shown great performance over a diverse set of tasks. However, little is known about their limitations, as the high dimensional space learned by th…
Phenaki: Variable Length Video Generation From Open Domain Textual Description
Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans +6
We present Phenaki, a model capable of realistic video synthesis, given a sequence of textual prompts. Generating videos from text is particularly challenging due to the computatio…
WildQA: In-the-Wild Video Question Answering
Santiago Castro, Naihao Deng, Pingxuan Huang +2
Existing video understanding datasets mostly focus on human interactions, with little attention being paid to the "in the wild" settings, where the videos are recorded outdoors. We…
FitCLIP: Refining Large-Scale Pretrained Image-Text Models for Zero-Shot Video Understanding Tasks
Santiago Castro, Fabian Caba Heilbron
Large-scale pretrained image-text models have shown incredible zero-shot performance in a handful of tasks, including video ones such as action recognition and text-to-video retrie…
When Did It Happen? Duration-informed Temporal Localization of Narrated Actions in Vlogs
Oana Ignat, Santiago Castro, Yuhang Zhou +3
We consider the task of temporal human action localization in lifestyle vlogs. We introduce a novel dataset consisting of manual annotations of temporal localization for 13,000 nar…
WhyAct: Identifying Action Reasons in Lifestyle Vlogs
Oana Ignat, Santiago Castro, Hanwen Miao +2
We aim to automatically identify human action reasons in online videos. We focus on the widespread genre of lifestyle vlogs, in which people perform actions while verbally describi…
FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation Framework
Santiago Castro, Ruoyao Wang, Pingxuan Huang +5
We propose fill-in-the-blanks as a video understanding evaluation framework and introduce FIBER -- a novel dataset consisting of 28,000 videos and descriptions in support of this e…