21 citations · 77 across the 24 of their papers we have counts for
30 papers
GaussianHeadTalk: Wobble-Free 3D Talking Heads with Audio Driven Gaussian Splatting
Madhav Agarwal, Mingtian Zhang, Laura Sevilla-Lara +1
Speech-driven talking heads have recently emerged and enable interactive avatars. However, real-world applications are limited, as current methods achieve high visual fidelity but…
Mask2IV: Interaction-Centric Video Generation via Mask Trajectories
Gen Li, Bo Zhao, Jianfei Yang +1
Generating interaction-centric videos, such as those depicting humans or robots interacting with objects, is crucial for embodied intelligence, as they provide rich and diverse vis…
Predicting Implicit Arguments in Procedural Video Instructions
Anil Batra, Laura Sevilla-Lara, Marcus Rohrbach +1
Procedural texts help AI enhance reasoning about context and action sequences. Transforming these into Semantic Role Labeling (SRL) improves understanding of individual steps by id…
Progressive Data Dropout: An Embarrassingly Simple Approach to Faster Training
Shriram M Sathiyanarayanan, Xinyue Hao, Shihao Hou +4
The success of the machine learning field has reliably depended on training on large datasets. While effective, this trend comes at an extraordinary cost. This is due to two deeply…
Principles of Visual Tokens for Efficient Video Understanding
Xinyue Hao, Gen Li, Shreyank N Gowda +4
Video understanding has made huge strides in recent years, relying largely on the power of transformers. As this architecture is notoriously expensive and video data is highly redu…
Continual Learning Improves Zero-Shot Action Recognition
Shreyank N Gowda, Davide Moltisanti, Laura Sevilla-Lara
Zero-shot action recognition requires a strong ability to generalize from pre-training and seen classes to novel unseen classes. Similarly, continual learning aims to develop model…
Learning Precise Affordances from Egocentric Videos for Robotic Manipulation
Gen Li, Nikolaos Tsagkas, Jifei Song +4
Affordance, defined as the potential actions that an object offers, is crucial for embodied AI agents. For example, such knowledge directs an agent to grasp a knife by the handle f…
Coarse or Fine? Recognising Action End States without Labels
Davide Moltisanti, Hakan Bilen, Laura Sevilla-Lara +1
We focus on the problem of recognising the end state of an action in an image, which is critical for understanding what action is performed and in which manner. We study this focus…
One-Shot Open Affordance Learning with Foundation Models
Gen Li, Deqing Sun, Laura Sevilla-Lara +1
We introduce One-shot Open Affordance Learning (OOAL), where a model is trained with just one example per base object category, but is expected to identify novel objects and afford…
Efficient Pre-training for Localized Instruction Generation of Videos
Anil Batra, Davide Moltisanti, Laura Sevilla-Lara +2
Procedural videos, exemplified by recipe demonstrations, are instrumental in conveying step-by-step instructions. However, understanding such videos is challenging as it involves t…
Watt For What: Rethinking Deep Learning's Energy-Performance Relationship
Shreyank N Gowda, Xinyue Hao, Gen Li +3
Deep learning models have revolutionized various fields, from image recognition to natural language processing, by achieving unprecedented levels of accuracy. However, their increa…
Telling Stories for Common Sense Zero-Shot Action Recognition
Shreyank N Gowda, Laura Sevilla-Lara
Video understanding has long suffered from reliance on large labeled datasets, motivating research into zero-shot learning. Recent progress in language modeling presents opportunit…
LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Grounding
Gen Li, Varun Jampani, Deqing Sun +1
Humans excel at acquiring knowledge through observation. For example, we can learn to use new tools by watching demonstrations. This skill is fundamental for intelligent systems to…
Learning Action Changes by Measuring Verb-Adverb Textual Relationships
Davide Moltisanti, Frank Keller, Hakan Bilen +1
The goal of this work is to understand the way actions are performed in videos. That is, given a video, we aim to predict an adverb indicating a modification applied to the action…
An Action Is Worth Multiple Words: Handling Ambiguity in Action Recognition
Kiyoon Kim, Davide Moltisanti, Oisin Mac Aodha +1
Precisely naming the action depicted in a video can be a challenging and oftentimes ambiguous task. In contrast to object instances represented as nouns (e.g. dog, cat, chair, etc.…
A Closer Look at Temporal Ordering in the Segmentation of Instructional Videos
Anil Batra, Shreyank N Gowda, Frank Keller +1
Understanding the steps required to perform a task is an important skill for AI systems. Learning these steps from instructional videos involves two subproblems: (i) identifying th…
Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognition
Shreyank N Gowda, Marcus Rohrbach, Frank Keller +1
We address the problem of data augmentation for video action recognition. Standard augmentation strategies in video are hand-designed and sample the space of possible augmented dat…
Capturing Temporal Information in a Single Frame: Channel Sampling Strategies for Action Recognition
Kiyoon Kim, Shreyank N Gowda, Oisin Mac Aodha +1
We address the problem of capturing temporal information for video classification in 2D networks, without increasing their computational cost. Existing approaches focus on modifyin…
A New Split for Evaluating True Zero-Shot Action Recognition
Shreyank N Gowda, Laura Sevilla-Lara, Kiyoon Kim +2
Zero-shot action recognition is the task of classifying action categories that are not available in the training set. In this setting, the standard evaluation protocol is to use ex…
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation
Gen Li, Varun Jampani, Laura Sevilla-Lara +3
Prototype learning is extensively used for few-shot segmentation. Typically, a single prototype is obtained from the support feature by averaging the global object information. How…
CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition
Shreyank N Gowda, Laura Sevilla-Lara, Frank Keller +1
Zero-shot action recognition is the task of recognizingaction classes without visual examples, only with a seman-tic embedding which relates unseen to seen classes. Theproblem can…
SMART Frame Selection for Action Recognition
Shreyank N Gowda, Marcus Rohrbach, Laura Sevilla-Lara
Action recognition is computationally expensive. In this paper, we address the problem of frame selection to improve the accuracy of action recognition. In particular, we show that…
Proceedings of the ICLR Workshop on Computer Vision for Agriculture (CV4A) 2020
Yannis Kalantidis, Laura Sevilla-Lara, Ernest Mwebaze +3
This is the proceedings of the Computer Vision for Agriculture (CV4A) Workshop that was held in conjunction with the International Conference on Learning Representations (ICLR) 202…
ALBA : Reinforcement Learning for Video Object Segmentation
Shreyank N Gowda, Panagiotis Eustratiadis, Timothy Hospedales +1
We consider the challenging problem of zero-shot video object segmentation (VOS). That is, segmenting and tracking multiple moving objects within a video fully automatically, witho…