80 citations · 136 across the 10 of their papers we have counts for
12 papers
Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Prediction
Oana Ignat, Santiago Castro, Weiji Li +1
We introduce the task of automatic human action co-occurrence identification, i.e., determine whether two human actions can co-occur in the same interval of time. We create and mak…
Scalable Performance Analysis for Vision-Language Models
Santiago Castro, Oana Ignat, Rada Mihalcea
Joint vision-language models have shown great performance over a diverse set of tasks. However, little is known about their limitations, as the high dimensional space learned by th…
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models
Oana Ignat, Zhijing Jin, Artem Abzaliev +19
Recent progress in large language models (LLMs) has enabled the deployment of many generative NLP applications. At the same time, it has also led to a misleading public discourse t…
Phenaki: Variable Length Video Generation From Open Domain Textual Description
Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans +6
We present Phenaki, a model capable of realistic video synthesis, given a sequence of textual prompts. Generating videos from text is particularly challenging due to the computatio…
WildQA: In-the-Wild Video Question Answering
Santiago Castro, Naihao Deng, Pingxuan Huang +2
Existing video understanding datasets mostly focus on human interactions, with little attention being paid to the "in the wild" settings, where the videos are recorded outdoors. We…
FitCLIP: Refining Large-Scale Pretrained Image-Text Models for Zero-Shot Video Understanding Tasks
Santiago Castro, Fabian Caba Heilbron
Large-scale pretrained image-text models have shown incredible zero-shot performance in a handful of tasks, including video ones such as action recognition and text-to-video retrie…
When Did It Happen? Duration-informed Temporal Localization of Narrated Actions in Vlogs
Oana Ignat, Santiago Castro, Yuhang Zhou +3
We consider the task of temporal human action localization in lifestyle vlogs. We introduce a novel dataset consisting of manual annotations of temporal localization for 13,000 nar…
WhyAct: Identifying Action Reasons in Lifestyle Vlogs
Oana Ignat, Santiago Castro, Hanwen Miao +2
We aim to automatically identify human action reasons in online videos. We focus on the widespread genre of lifestyle vlogs, in which people perform actions while verbally describi…
FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation Framework
Santiago Castro, Ruoyao Wang, Pingxuan Huang +5
We propose fill-in-the-blanks as a video understanding evaluation framework and introduce FIBER -- a novel dataset consisting of 28,000 videos and descriptions in support of this e…
Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)
Santiago Castro, Devamanyu Hazarika, Verónica Pérez-Rosas +3
Sarcasm is often expressed through several verbal and non-verbal cues, e.g., a change of tone, overemphasis in a word, a drawn-out syllable, or a straight looking face. Most of the…
RETUYT in TASS 2017: Sentiment Analysis for Spanish Tweets using SVM and CNN
Aiala Rosá, Luis Chiruzzo, Mathias Etcheverry +1
This article presents classifiers based on SVM and Convolutional Neural Networks (CNN) for the TASS 2017 challenge on tweets sentiment analysis. The classifier with the best perfor…
Is This a Joke? Detecting Humor in Spanish Tweets
Santiago Castro, Matías Cubero, Diego Garat +1
While humor has been historically studied from a psychological, cognitive and linguistic standpoint, its study from a computational perspective is an area yet to be explored in Com…