128 citations · 398 across the 20 of their papers we have counts for
23 papers
InstrAct: Towards Action-Centric Understanding in Instructional Videos
Zhuoyi Yang, Jiapeng Yu, Reuben Tan +2
Understanding instructional videos requires recognizing fine-grained actions and modeling their temporal relations, which remains challenging for current Video Foundation Models (V…
CircleNet: Reciprocating Feature Adaptation for Robust Pedestrian Detection
Tianliang Zhang, Zhenjun Han, Huijuan Xu +2
Pedestrian detection in the wild remains a challenging problem especially when the scene contains significant occlusion and/or low resolution of the pedestrians to be detected. Exi…
Weakly-Supervised Temporal Action Detection for Fine-Grained Videos with Hierarchical Atomic Actions
Zhi Li, Lu He, Huijuan Xu
Action understanding has evolved into the era of fine granularity, as most human behaviors in real life have only minor differences. To detect these fine-grained actions accurately…
Disentangled Action Recognition with Knowledge Bases
Zhekun Luo, Shalini Ghosh, Devin Guillory +3
Action in video usually involves the interaction of human with objects. Action labels are typically composed of various combinations of verbs and nouns, but we may not have trainin…
Syntax Controlled Knowledge Graph-to-Text Generation with Order and Semantic Consistency
Jin Liu, Chongfeng Fan, Fengyu Zhou +1
The knowledge graph (KG) stores a large amount of structural knowledge, while it is not easy for direct human understanding. Knowledge graph-to-text (KG-to-text) generation aims to…
Temporal Action Detection with Multi-level Supervision
Baifeng Shi, Qi Dai, Judy Hoffman +3
Training temporal action detection in videos requires large amounts of labeled data, yet such annotation is expensive to collect. Incorporating unlabeled or weakly-labeled data to…
Auxiliary Task Reweighting for Minimum-data Learning
Baifeng Shi, Judy Hoffman, Kate Saenko +2
Supervised learning requires a large amount of training data, limiting its application where labeled data is scarce. To compensate for data scarcity, one possible method is to util…
Spatio-Temporal Action Detection with Multi-Object Interaction
Huijuan Xu, Lizhi Yang, Stan Sclaroff +2
Spatio-temporal action detection in videos requires localizing the action both spatially and temporally in the form of an "action tube". Nowadays, most spatio-temporal action detec…
Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning
Zhekun Luo, Devin Guillory, Baifeng Shi +4
Weakly-supervised action localization requires training a model to localize the action segments in the video given only video level action label. It can be solved under the Multipl…
Revisiting Few-shot Activity Detection with Class Similarity Control
Huijuan Xu, Ximeng Sun, Eric Tzeng +3
Many interesting events in the real world are rare making preannotated machine learning ready videos a rarity in consequence. Thus, temporal activity detection models that are able…
Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks
Joanna Materzynska, Tete Xiao, Roei Herzig +3
Human action is naturally compositional: humans can easily recognize and perform actions with objects that are different from those used in training demonstrations. In this paper,…
Learning Canonical Representations for Scene Graph to Image Generation
Roei Herzig, Amir Bar, Huijuan Xu +3
Generating realistic images of complex visual scenes becomes challenging when one wishes to control the structure of the generated images. Previous approaches showed that scenes wi…
LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval
Reuben Tan, Huijuan Xu, Kate Saenko +1
The goal of weakly-supervised video moment retrieval is to localize the video segment most relevant to the given natural language query without access to temporal annotations durin…
Two-Stream Region Convolutional 3D Network for Temporal Activity Detection
Huijuan Xu, Abir Das, Kate Saenko
We address the problem of temporal activity detection in continuous, untrimmed video streams. This is a difficult task that requires extracting meaningful spatio-temporal features…
Similarity R-C3D for Few-shot Temporal Activity Detection
Huijuan Xu, Bingyi Kang, Ximeng Sun +3
Many activities of interest are rare events, with only a few labeled examples available. Therefore models for temporal activity detection which are able to learn from a few example…
Spatio-Temporal Action Graph Networks
Roei Herzig, Elad Levi, Huijuan Xu +5
Events defined by the interaction of objects in a scene are often of critical importance; yet important events may have insufficient labeled examples to train a conventional deep m…
TwoStreamVAN: Improving Motion Modeling in Video Generation
Ximeng Sun, Huijuan Xu, Kate Saenko
Video generation is an inherently challenging task, as it requires modeling realistic temporal dynamics as well as spatial content. Existing methods entangle the two intrinsically…
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Huijuan Xu, Kun He, Bryan A. Plummer +3
We address the problem of text-based activity retrieval in video. Given a sentence describing an activity, our task is to retrieve matching clips from an untrimmed video. To captur…
Joint Event Detection and Description in Continuous Video Streams
Huijuan Xu, Boyang Li, Vasili Ramanishka +2
Dense video captioning is a fine-grained video understanding task that involves two sub-problems: localizing distinct events in a long video stream, and generating captions for the…
Contextual Multi-Scale Region Convolutional 3D Network for Activity Detection
Yancheng Bai, Huijuan Xu, Kate Saenko +1
Activity detection is a fundamental problem in computer vision. Detecting activities of different temporal scales is particularly challenging. In this paper, we propose the context…
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
Huijuan Xu, Abir Das, Kate Saenko
We address the problem of activity detection in continuous, untrimmed video streams. This is a difficult task that requires extracting meaningful spatio-temporal features to captur…
Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
Huijuan Xu, Kate Saenko
We address the problem of Visual Question Answering (VQA), which requires joint image and language understanding to answer a question about a given photograph. Recent approaches ha…
A Multi-scale Multiple Instance Video Description Network
Huijuan Xu, Subhashini Venugopalan, Vasili Ramanishka +2
Generating natural language descriptions for in-the-wild videos is a challenging task. Most state-of-the-art methods for solving this problem borrow existing deep convolutional neu…