1k citations · 1.8k across the 49 of their papers we have counts for
53 papers
Training Vision-Language Transformers from Captions
Liangke Gui, Yingshan Chang, Qiuyuan Huang +4
Vision-Language Transformers can be learned without low-level human labels (e.g. class labels, bounding boxes, etc). Existing work, whether explicitly utilizing bounding boxes or p…
Spatial-Temporal Alignment Network for Action Recognition and Detection
Junwei Liang, Liangliang Cao, Xuehan Xiong +2
This paper studies how to introduce viewpoint-invariant feature representations that can help action recognition and detection. Although we have witnessed great progress of action…
Event-Related Bias Removal for Real-time Disaster Events
Evangelia Spiliopoulou, Salvador Medina Maza, Eduard Hovy +1
Social media has become an important tool to share information about crisis events such as natural disasters and mass attacks. Detecting actionable posts that contain useful inform…
Support-set bottlenecks for video-text representation learning
Mandela Patrick, Po-Yao Huang, Yuki Asano +4
The dominant paradigm for learning video-text representations -- noise contrastive learning -- increases the similarity of the representations of pairs of samples that are known to…
Robust Long-Term Object Tracking via Improved Discriminative Model Prediction
Seokeon Choi, Junhyun Lee, Yunsung Lee +1
We propose an improved discriminative model prediction method for robust long-term tracking based on a pre-trained short-term tracker. The baseline pre-trained short-term tracker i…
From A Glance to "Gotcha": Interactive Facial Image Retrieval with Progressive Relevance Feedback
Xinru Yang, Haozhi Qi, Mingyang Li +1
Facial image retrieval plays a significant role in forensic investigations where an untrained witness tries to identify a suspect from a massive pool of images. However, due to the…
MSNet: A Multilevel Instance Segmentation Network for Natural Disaster Damage Assessment in Aerial Videos
Xiaoyu Zhu, Junwei Liang, Alexander Hauptmann
In this paper, we study the problem of efficiently assessing building damage after natural disasters like hurricanes, floods or fires, through aerial video analysis. We make two ma…
Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
Po-Yao Huang, Junjie Hu, Xiaojun Chang +1
Unsupervised machine translation (MT) has recently achieved impressive results with monolingual corpora only. However, it is still challenging to associate source-target sentences…
SimAug: Learning Robust Representations from Simulation for Trajectory Prediction
Junwei Liang, Lu Jiang, Alexander Hauptmann
This paper studies the problem of predicting future trajectories of people in unseen cameras of novel scenarios and views. We approach this problem through the real-data-free setti…
ZSTAD: Zero-Shot Temporal Activity Detection
Lingling Zhang, Xiaojun Chang, Jun Liu +4
An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos. Curren…
Gun Source and Muzzle Head Detection
Zhong Zhou, Isak Czeresnia Etinger, Florian Metze +2
There is a surging need across the world for protection against gun violence. There are three main areas that we have identified as challenging in research that tries to curb gun v…
The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
Junwei Liang, Lu Jiang, Kevin Murphy +2
This paper studies the problem of predicting the distribution over multiple possible future paths of people as they move through various visual scenes. We make two main contributio…
Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations
Po-Yao Huang, Xiaojun Chang, Alexander Hauptmann
With the aim of promoting and understanding the multilingual version of image search, we leverage visual object detection and propose a model with diverse multi-head attention to l…
Improving the Learning of Multi-column Convolutional Neural Network for Crowd Counting
Zhi-Qi Cheng, Jun-Xiu Li, Qi Dai +3
Tremendous variation in the scale of people/head size is a critical problem for crowd counting. To improve the scale invariance of feature representation, recent works extensively…
Learning Spatial Awareness to Improve Crowd Counting
Zhi-Qi Cheng, Jun-Xiu Li, Qi Dai +2
The aim of crowd counting is to estimate the number of people in images by leveraging the annotation of center positions for pedestrians' heads. Promising progresses have been made…
Report of 2017 NSF Workshop on Multimedia Challenges, Opportunities and Research Roadmaps
Shih-Fu Chang, Alex Hauptmann, Louis-Philippe Morency +18
With the transformative technologies and the rapidly changing global R&D landscape, the multimedia and multimodal community is now faced with many new opportunities and uncertainti…
Activitynet 2019 Task 3: Exploring Contexts for Dense Captioning Events in Videos
Shizhe Chen, Yuqing Song, Yida Zhao +5
Contextual reasoning is essential to understand events in long untrimmed videos. In this work, we systematically explore different captioning models with various contexts for the d…
Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data
Shizhe Chen, Qin Jin, Alexander Hauptmann
Bilingual lexicon induction, translating words from the source language to the target language, is a long-standing natural language processing task. Recent endeavors prove that it…
Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond
Junwei Liang, Jay D. Aronson, Alexander Hauptmann
Every minute, hundreds of hours of video are uploaded to social media sites and the Internet from around the world. This material creates a visual record of the experiences of a si…
ExCL: Extractive Clip Localization Using Natural Language Descriptions
Soham Ghosh, Anuva Agarwal, Zarana Parekh +1
The task of retrieving clips within videos based on a given natural language query requires cross-modal reasoning over multiple frames. Prior approaches such as sliding window clas…
Contrastive Adaptation Network for Unsupervised Domain Adaptation
Guoliang Kang, Lu Jiang, Yi Yang +1
Unsupervised Domain Adaptation (UDA) makes predictions for the target domain data while manual annotations are only available in the source domain. Previous methods minimize the do…
Perceiving Physical Equation by Observing Visual Scenarios
Siyu Huang, Zhi-Qi Cheng, Xi Li +3
Inferring universal laws of the environment is an important ability of human intelligence as well as a symbol of general AI. In this paper, we take a step toward this goal such tha…
Traffic Danger Recognition With Surveillance Cameras Without Training Data
Lijun Yu, Dawei Zhang, Xiangqun Chen +1
We propose a traffic danger recognition model that works with arbitrary traffic surveillance cameras to identify and predict car crashes. There are too many cameras to monitor manu…
Learning Sound Events From Webly Labeled Data
Anurag Kumar, Ankit Shah, Bhiksha Raj +1
In the last couple of years, weakly labeled learning has turned out to be an exciting approach for audio event detection. In this work, we introduce webly labeled learning for soun…