406 citations · 3.1k across the 87 of their papers we have counts for
60 papers · 1 filter
Few-Shot Object Detection with Fully Cross-Transformer
Guangxing Han, Jiawei Ma, Shiyuan Huang +2
Few-shot object detection (FSOD), with the aim to detect novel objects using very few training examples, has recently attracted great research interest in the community. Metric-lea…
Learning To Recognize Procedural Activities with Distant Supervision
Xudong Lin, Fabio Petroni, Gedas Bertasius +3
In this paper we consider the problem of classifying fine-grained, multi-step activities (e.g., cooking different recipes, making disparate home improvements, creating various form…
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks
Zhecan Wang, Noel Codella, Yen-Chun Chen +7
Contrastive language-image pretraining (CLIP) links vision and language modalities into a unified embedding space, yielding the tremendous potential for vision-language (VL) tasks.…
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
Zhecan Wang, Haoxuan You, Liunian Harold Li +5
Answering complex questions about images is an ambitious goal for machine intelligence, which requires a joint understanding of images, text, and commonsense knowledge, as well as…
Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networks
Guangxing Han, Yicheng He, Shiyuan Huang +2
Few-shot object detection (FSOD) aims to detect never-seen objects using few examples. This field sees recent improvement owing to the meta-learning techniques by learning how to m…
PreViTS: Contrastive Pretraining with Video Tracking Supervision
Brian Chen, Ramprasaath R. Selvaraju, Shih-Fu Chang +2
Videos are a rich source for self-supervised learning (SSL) of visual representations due to the presence of natural temporal transformations of objects. However, current methods t…
Joint Multimedia Event Extraction from Video and Article
Brian Chen, Xudong Lin, Christopher Thomas +5
Visual and textual modalities contribute complementary information about events described in multimedia documents. Videos contain rich dynamics and detailed unfoldings of events, w…
Partner-Assisted Learning for Few-Shot Image Classification
Jiawei Ma, Hanchen Xie, Guangxing Han +3
Few-shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation. Even though the idea of meta-learn…
Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos
Brian Chen, Andrew Rouditchenko, Kevin Duarte +10
Multimodal self-supervised learning is getting more and more attention as it allows not only to train large networks without human supervision but also to search and retrieve data…
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
Hassan Akbari, Liangzhe Yuan, Rui Qian +4
We present a framework for learning multimodal representations from unlabeled data using convolution-free Transformer architectures. Specifically, our Video-Audio-Text Transformer…
Meta Faster R-CNN: Towards Accurate Few-Shot Object Detection with Attentive Feature Alignment
Guangxing Han, Shiyuan Huang, Jiawei Ma +2
Few-shot object detection (FSOD) aims to detect objects using only a few examples. How to adapt state-of-the-art object detectors to the few-shot domain remains challenging. Object…
Co-Grounding Networks with Semantic Attention for Referring Expression Comprehension in Videos
Sijie Song, Xudong Lin, Jiaying Liu +2
In this paper, we address the problem of referring expression comprehension in videos, which is challenging due to complex expression and scene dynamics. Unlike previous methods wh…
VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
Xudong Lin, Gedas Bertasius, Jue Wang +3
We present \textsc{Vx2Text}, a framework for text generation from multimodal inputs consisting of video plus text, speech, or audio. In order to leverage transformer networks, whic…
Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language
Hassan Akbari, Hamid Palangi, Jianwei Yang +6
Neuro-symbolic representations have proved effective in learning structure information in vision and language. In this paper, we propose a new model architecture for learning multi…
Open-Vocabulary Object Detection Using Captions
Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu +1
Despite the remarkable accuracy of deep neural networks in object detection, they are costly to train and scale due to supervision requirements. Particularly, learning more object…
Uncertainty-Aware Few-Shot Image Classification
Zhizheng Zhang, Cuiling Lan, Wenjun Zeng +2
Few-shot image classification learns to recognize new categories from limited labelled data. Metric learning based approaches have been widely investigated, where a query sample is…
Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding
Long Chen, Wenbo Ma, Jun Xiao +2
The prevailing framework for solving referring expression grounding is based on a two-stage process: 1) detecting proposals with an object detector and 2) grounding the referent to…
Beyond Triplet Loss: Meta Prototypical N-tuple Loss for Person Re-identification
Zhizheng Zhang, Cuiling Lan, Wenjun Zeng +2
Person Re-identification (ReID) aims at matching a person of interest across images. In convolutional neural network (CNN) based approaches, loss design plays a vital role in pulli…
Deep Learning Guided Building Reconstruction from Satellite Imagery-derived Point Clouds
Bo Xu, Xu Zhang, Zhixin Li +3
3D urban reconstruction of buildings from remotely sensed imagery has drawn significant attention during the past two decades. While aerial imagery and LiDAR provide higher resolut…
Unifying Specialist Image Embedding into Universal Image Embedding
Yang Feng, Futang Peng, Xu Zhang +7
Deep image embedding provides a way to measure the semantic similarity of two images. It plays a central role in many applications such as image search, face verification, and zero…
Weakly Supervised Visual Semantic Parsing
Alireza Zareian, Svebor Karaman, Shih-Fu Chang
Scene Graph Generation (SGG) aims to extract entities, predicates and their semantic structure from images, enabling deep understanding of visual content, with many applications su…
Bridging Knowledge Graphs to Generate Scene Graphs
Alireza Zareian, Svebor Karaman, Shih-Fu Chang
Scene graphs are powerful representations that parse images into their abstract semantic elements, i.e., objects and their interactions, which facilitates visual comprehension and…
General Partial Label Learning via Dual Bipartite Graph Autoencoder
Brian Chen, Bo Wu, Alireza Zareian +2
We formulate a practical yet challenging problem: General Partial Label Learning (GPLL). Compared to the traditional Partial Label Learning (PLL) problem, GPLL relaxes the supervis…
Flow-Distilled IP Two-Stream Networks for Compressed Video Action Recognition
Shiyuan Huang, Xudong Lin, Svebor Karaman +1
Two-stream networks have achieved great success in video recognition. A two-stream network combines a spatial stream of RGB frames and a temporal stream of Optical Flow to make pre…