915 citations · 1.9k across the 40 of their papers we have counts for
41 papers
AiluRus: A Scalable ViT Framework for Dense Prediction
Jin Li, Yaoming Wang, Xiaopeng Zhang +6
Vision transformers (ViTs) have emerged as a prevalent architecture for vision tasks owing to their impressive performance. However, when it comes to handling long token sequences,…
From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models
Dongsheng Jiang, Yuchen Liu, Songlin Liu +6
Multi-modal Large Language Models (MLLMs) have made significant strides in expanding the capabilities of Large Language Models (LLMs) through the incorporation of visual perception…
Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulation
Shuangrui Ding, Peisen Zhao, Xiaopeng Zhang +3
Transformers have become the primary backbone of the computer vision community due to their impressive performance. However, the unfriendly computation cost impedes their potential…
Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners
Bowen Shi, Xiaopeng Zhang, Yaoming Wang +5
Representation learning has been evolving from traditional supervised training to Contrastive Learning (CL) and Masked Image Modeling (MIM). Previous works have demonstrated their…
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models
Lingxi Xie, Longhui Wei, Xiaopeng Zhang +4
The AI community has been pursuing algorithms known as artificial general intelligence (AGI) that apply to any kind of real-world problem. Recently, chat systems powered by large l…
ControlVideo: Training-free Controllable Text-to-Video Generation
Yabo Zhang, Yuxiang Wei, Dongsheng Jiang +3
Text-driven diffusion models have unlocked unprecedented abilities in image generation, whereas their video counterpart still lags behind due to the excessive training cost of temp…
Segment Anything in 3D with Radiance Fields
Jiazhong Cen, Jiemin Fang, Zanwei Zhou +5
The Segment Anything Model (SAM) emerges as a powerful vision foundation model to generate high-quality 2D segmentation results. This paper aims to generalize SAM to segment 3D obj…
M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios
Ning Liao, Xiaopeng Zhang, Min Cao +1
In realistic open-set scenarios where labels of a part of testing data are totally unknown, when vision-language (VL) prompt learning methods encounter inputs related to unknown cl…
Rethinking Visual Prompt Learning as Masked Visual Token Modeling
Ning Liao, Bowen Shi, Xiaopeng Zhang +3
Prompt learning has achieved great success in efficiently exploiting large-scale pre-trained models in natural language processing (NLP). It reformulates the downstream tasks as th…
Feature Calibration Network for Occluded Pedestrian Detection
Tianliang Zhang, Qixiang Ye, Baochang Zhang +3
Pedestrian detection in the wild remains a challenging problem especially for scenes containing serious occlusion. In this paper, we propose a novel feature learning method in the…
Motion-inductive Self-supervised Object Discovery in Videos
Shuangrui Ding, Weidi Xie, Yabo Chen +4
In this paper, we consider the task of unsupervised object discovery in videos. Previous works have shown promising results via processing optical flows to segment objects. However…
SdAE: Self-distillated Masked Autoencoder
Yabo Chen, Yuchen Liu, Dongsheng Jiang +4
With the development of generative-based self-supervised learning (SSL) approaches like BeiT and MAE, how to learn good representations by masking random patches of the input image…
Active Pointly-Supervised Instance Segmentation
Chufeng Tang, Lingxi Xie, Gang Zhang +3
The requirement of expensive annotations is a major burden for training a well-performed instance segmentation model. In this paper, we present an economic active learning setting,…
Visual Recognition by Request
Chufeng Tang, Lingxi Xie, Xiaopeng Zhang +2
Humans have the ability of recognizing visual semantics in an unlimited granularity, but existing visual recognition algorithms cannot achieve this goal. In this paper, we establis…
Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers
Yunjie Tian, Lingxi Xie, Jiemin Fang +6
The past year has witnessed a rapid development of masked image modeling (MIM). MIM is mostly built upon the vision transformers, which suggests that self-supervised visual represe…
TAPE: Task-Agnostic Prior Embedding for Image Restoration
Lin Liu, Lingxi Xie, Xiaopeng Zhang +5
Learning a generalized prior for natural image restoration is an important yet challenging task. Early methods mostly involved handcrafted priors including normalized sparsity, l_0…
NeuSample: Neural Sample Field for Efficient View Synthesis
Jiemin Fang, Lingxi Xie, Xinggang Wang +3
Neural radiance fields (NeRF) have shown great potentials in representing 3D scenes and synthesizing novel views, but the computational overhead of NeRF at the inference stage is s…
Semantic-Aware Generation for Self-Supervised Visual Representation Learning
Yunjie Tian, Lingxi Xie, Xiaopeng Zhang +6
In this paper, we propose a self-supervised visual representation learning approach which involves both generative and discriminative proxies, where we focus on the former part by…
Bag of Instances Aggregation Boosts Self-supervised Distillation
Haohang Xu, Jiemin Fang, Xiaopeng Zhang +5
Recent advances in self-supervised learning have experienced remarkable progress, especially for contrastive learning based methods, which regard each image as well as its augmenta…
Multi-dataset Pretraining: A Unified Model for Semantic Segmentation
Bowen Shi, Xiaopeng Zhang, Haohang Xu +4
Collecting annotated data for semantic segmentation is time-consuming and hard to scale up. In this paper, we for the first time propose a unified framework, termed as Multi-Datase…
What Is Considered Complete for Visual Recognition?
Lingxi Xie, Xiaopeng Zhang, Longhui Wei +2
This is an opinion paper. We hope to deliver a key message that current visual recognition systems are far from complete, i.e., recognizing everything that human can recognize, yet…
Semi-supervised Contrastive Learning with Similarity Co-calibration
Yuhang Zhang, Xiaopeng Zhang, Robert. C. Qiu +3
Semi-supervised learning acts as an effective way to leverage massive unlabeled data. In this paper, we propose a novel training strategy, termed as Semi-supervised Contrastive Lea…
Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
Hu Cao, Yueyue Wang, Joy Chen +4
In the past few years, convolutional neural networks (CNNs) have achieved milestones in medical image analysis. Especially, the deep neural networks based on U-shaped architecture…
MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokens
Jiemin Fang, Lingxi Xie, Xinggang Wang +3
Transformers have offered a new methodology of designing neural networks for visual recognition. Compared to convolutional networks, Transformers enjoy the ability of referring to…