349 citations · 519 across the 27 of their papers we have counts for
8 papers · 1 filter
Video Recognition in Portrait Mode
Mingfei Han, Linjie Yang, Xiaojie Jin +3
The creation of new datasets often presents new challenges for video recognition and can inspire novel ideas while addressing these challenges. While existing datasets mainly compr…
Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos
Mingfei Han, Linjie Yang, Xiaojun Chang +2
A short clip of video may contain progression of multiple events and an interesting story line. A human need to capture both the event in every shot and associate them together to…
Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling
Haogeng Liu, Qihang Fan, Tingkai Liu +5
This paper proposes Video-Teller, a video-language foundation model that leverages multi-modal fusion and fine-grained modality alignment to significantly enhance the video-to-text…
Selective Feature Adapter for Dense Vision Transformers
Xueqing Deng, Qi Fan, Xiaojie Jin +2
Fine-tuning pre-trained transformer models, e.g., Swin Transformer, are successful in numerous downstream for dense prediction vision tasks. However, one major issue is the cost/st…
The Devil is in the Details: A Deep Dive into the Rabbit Hole of Data Filtering
Haichao Yu, Yu Tian, Sateesh Kumar +2
The quality of pre-training data plays a critical role in the performance of foundation models. Popular foundation models often design their own recipe for data filtering, which ma…
Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?
Cheng-En Wu, Yu Tian, Haichao Yu +4
Vision-language models such as CLIP learn a generic text-image embedding from large-scale training data. A vision-language model can be adapted to a new classification task through…
Exploring the Role of Audio in Video Captioning
Yuhan Shen, Linjie Yang, Longyin Wen +3
Recent focus in video captioning has been on designing architectures that can consume both video and text modalities, and using large-scale video datasets with text transcripts for…
Former: Unified etrieval and eranking Transformer for Place Recognition
Sijie Zhu, Linjie Yang, Chen Chen +3
Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database. Conventional methods generally adopt aggregated CNN feat…