406 citations · 1.8k across the 130 of their papers we have counts for
22 papers · 1 filter
Rethinking Nearest Neighbors for Visual Classification
Menglin Jia, Bor-Chun Chen, Zuxuan Wu +3
Neural network classifiers have become the de-facto choice for current "pre-train then fine-tune" paradigms of visual classification. In this paper, we investigate k-Nearest-Neighb…
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation
Tianyi Liu, Zuxuan Wu, Wenhan Xiong +2
Most existing vision-language pre-training methods focus on understanding tasks and use BERT-like objectives (masked language modeling and image-text matching) during pretraining.…
Cross-Modal Transferable Adversarial Attacks from Images to Videos
Zhipeng Wei, Jingjing Chen, Zuxuan Wu +1
Recent studies have shown that adversarial examples hand-crafted on one white-box model can be used to attack other black-box models. Such cross-model transferability makes it feas…
BEVT: BERT Pretraining of Video Transformers
Rui Wang, Dongdong Chen, Zuxuan Wu +6
This paper studies the BERT pretraining of video transformers. It is a straightforward but worth-studying extension given the recent success from BERT pretraining of image transfor…
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
Lingchen Meng, Hengduo Li, Bor-Chun Chen +4
Built on top of self-attention mechanisms, vision transformers have demonstrated remarkable performance on a variety of vision tasks recently. While achieving excellent performance…
Efficient Video Transformers with Spatial-Temporal Token Selection
Junke Wang, Xitong Yang, Hengduo Li +3
Video transformers have achieved impressive results on major video recognition benchmarks, which however suffer from high computational cost. In this paper, we present STTS, a toke…
Semi-Supervised Vision Transformers
Zejia Weng, Xitong Yang, Ang Li +2
We study the training of Vision Transformers for semi-supervised image classification. Transformers have recently demonstrated impressive performance on a multitude of supervised l…
Attacking Video Recognition Models with Bullet-Screen Comments
Kai Chen, Zhipeng Wei, Jingjing Chen +2
Recent research has demonstrated that Deep Neural Networks (DNNs) are vulnerable to adversarial patches which introduce perceptible but localized changes to the input. Nevertheless…
Boosting the Transferability of Video Adversarial Examples via Temporal Translation
Zhipeng Wei, Jingjing Chen, Zuxuan Wu +1
Although deep-learning based video recognition models have achieved remarkable success, they are vulnerable to adversarial examples that are generated by adding human-imperceptible…
Self-supervised Learning for Semi-supervised Temporal Language Grounding
Fan Luo, Shaoxiang Chen, Jingjing Chen +2
Given a text description, Temporal Language Grounding (TLG) aims to localize temporal boundaries of the segments that contain the specified semantics in an untrimmed video. TLG is…
Towards Transferable Adversarial Attacks on Vision Transformers
Zhipeng Wei, Jingjing Chen, Micah Goldblum +3
Vision transformers (ViTs) have demonstrated impressive performance on a series of computer vision tasks, yet they still suffer from adversarial examples. % crafted in a similar fa…
A Multimodal Framework for Video Ads Understanding
Zejia Weng, Lingchen Meng, Rui Wang +2
There is a growing trend in placing video advertisements on social platforms for online marketing, which demands automatic approaches to understand the contents of advertisements e…
FT-TDR: Frequency-guided Transformer and Top-Down Refinement Network for Blind Face Inpainting
Junke Wang, Shaoxiang Chen, Zuxuan Wu +1
Blind face inpainting refers to the task of reconstructing visual contents without explicitly indicating the corrupted regions in a face image. Inherently, this task faces two chal…
Cross-domain Contrastive Learning for Unsupervised Domain Adaptation
Rui Wang, Zuxuan Wu, Zejia Weng +3
Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain. Most existing UDA methods learn d…
Rethinking Pseudo Labels for Semi-Supervised Object Detection
Hengduo Li, Zuxuan Wu, Abhinav Shrivastava +1
Recent advances in semi-supervised object detection (SSOD) are largely driven by consistency-based pseudo-labeling methods for image classification tasks, producing pseudo labels a…
VideoLT: Large-scale Long-tailed Video Recognition
Xing Zhang, Zuxuan Wu, Zejia Weng +4
Label distributions in real-world are oftentimes long-tailed and imbalanced, resulting in biased models towards dominant labels. While long-tailed recognition has been extensively…
M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers
Tianrui Guan, Jun Wang, Shiyi Lan +4
We present a novel architecture for 3D object detection, M3DeTR, which combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales bas…
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
Junke Wang, Zuxuan Wu, Wenhao Ouyang +4
The widespread dissemination of Deepfakes demands effective approaches that can detect perceptually convincing forged images. In this paper, we aim to capture the subtle manipulati…
HCMS: Hierarchical and Conditional Modality Selection for Efficient Video Recognition
Zejia Weng, Zuxuan Wu, Hengduo Li +2
Videos are multimodal in nature. Conventional video recognition pipelines typically fuse multimodal features for improved performance. However, this is not only computationally exp…
Exploring Visual Engagement Signals for Representation Learning
Menglin Jia, Zuxuan Wu, Austin Reiter +3
Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues a…
THAT: Two Head Adversarial Training for Improving Robustness at Scale
Zuxuan Wu, Tom Goldstein, Larry S. Davis +1
Many variants of adversarial training have been proposed, with most research focusing on problems with relatively few classes. In this paper, we propose Two Head Adversarial Traini…
Deep Video Inpainting Detection
Peng Zhou, Ning Yu, Zuxuan Wu +3
This paper studies video inpainting detection, which localizes an inpainted region in a video both spatially and temporally. In particular, we introduce VIDNet, Video Inpainting De…