429 citations · 2.5k across the 69 of their papers we have counts for
72 papers
Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attacks
Jindong Gu, Fangyun Wei, Philip Torr +1
Deep Neural Networks can be easily fooled by small and imperceptible perturbations. The query-based black-box attack (QBBA) is able to create the perturbations using model output p…
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks
Zigang Geng, Binxin Yang, Tiankai Hang +8
We present InstructDiffusion, a unifying and generic framework for aligning computer vision tasks with human instructions. Unlike existing approaches that integrate prior knowledge…
Mask-Attention-Free Transformer for 3D Instance Segmentation
Xin Lai, Yuhui Yuan, Ruihang Chu +3
Recently, transformer-based methods have dominated 3D instance segmentation, where mask attention is commonly involved. Specifically, object queries are guided by the initial insta…
PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning
Mengya Han, Heliang Zheng, Chaoyue Wang +4
In this work, we address the task of few-shot part segmentation, which aims to segment the different parts of an unseen object using very few labeled examples. It is found that lev…
Rethinking the Localization in Weakly Supervised Object Localization
Rui Xu, Yong Luo, Han Hu +3
Weakly supervised object localization (WSOL) is one of the most popular and challenging tasks in computer vision. This task is to localize the objects in the images given only the…
V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection
Yichao Shen, Zigang Geng, Yuhui Yuan +6
We introduce a highly performant 3D object detector for point clouds using the DETR framework. The prior attempts all end up with suboptimal results because they fail to learn accu…
DETR Doesn't Need Multi-Scale or Locality Design
Yutong Lin, Yuhui Yuan, Zheng Zhang +3
This paper presents an improved DETR detector that maintains a "plain" nature: using a single-scale feature map and global cross-attention calculations without specific locality co…
LGViT: Dynamic Early Exiting for Accelerating Vision Transformer
Guanyu Xu, Jiawei Hao, Li Shen +4
Recently, the efficient deployment and acceleration of powerful vision transformers (ViTs) on resource-limited edge devices for providing multimedia services have become attractive…
Multi-Granularity Hand Action Detection
Ting Zhe, Jing Zhang, Yongqian Li +3
Detecting hand actions in videos is crucial for understanding video content and has diverse real-world applications. Existing approaches often focus on whole-body actions or coarse…
Improving Heterogeneous Model Reuse by Density Estimation
Anke Tang, Yong Luo, Han Hu +5
This paper studies multiparty learning, aiming to learn a model using the private data of different participants. Model reuse is a promising solution for multiparty learning, assum…
Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking
Xin Chen, Ben Kang, Jiawen Zhu +3
In this paper, we introduce a new sequence-to-sequence learning framework for RGB-based and multi-modal object tracking. First, we present SeqTrack for RGB-based tracking. It casts…
Side Adapter Network for Open-Vocabulary Semantic Segmentation
Mengde Xu, Zheng Zhang, Fangyun Wei +2
This paper presents a new framework for open-vocabulary semantic segmentation with the pre-trained vision-language model, named Side Adapter Network (SAN). Our approach models the…
Human Pose as Compositional Tokens
Zigang Geng, Chunyu Wang, Yixuan Wei +3
Human pose is typically represented by a coordinate vector of body joints or their heatmap embeddings. While easy for data processing, unrealistic pose estimates are admitted due t…
DeepMIM: Deep Supervision for Masked Image Modeling
Sucheng Ren, Fangyun Wei, Samuel Albanie +2
Deep supervision, which involves extra supervisions to the intermediate features of a neural network, was widely used in image classification in the early deep learning era since i…
SGDA: Towards 3D Universal Pulmonary Nodule Detection via Slice Grouped Domain Attention
Rui Xu, Zhi Liu, Yong Luo +5
Lung cancer is the leading cause of cancer death worldwide. The best solution for lung cancer is to diagnose the pulmonary nodules in the early stage, which is usually accomplished…
Efficient Diffusion Training via Min-SNR Weighting Strategy
Tiankai Hang, Shuyang Gu, Chen Li +5
Denoising diffusion models have been a mainstream approach for image generation, however, training these models often suffers from slow convergence. In this paper, we discovered th…
Subspace based Federated Unlearning
Guanghao Li, Li Shen, Yan Sun +3
Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data. Federated unlearning is an inverse FL proces…
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
Jia Ning, Chen Li, Zheng Zhang +4
Unlike language tasks, where the output space is usually limited to a set of tokens, the output space of visual tasks is more complicated, making it difficult to build a unified vi…
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
Sucheng Ren, Fangyun Wei, Zheng Zhang +1
Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only m…
Exploring Discrete Diffusion Models for Image Captioning
Zixin Zhu, Yixuan Wei, Jianfeng Wang +7
The image captioning task is typically realized by an auto-regressive method that decodes the text tokens one by one. We present a diffusion-based captioning model, dubbed the name…
Attentive Mask CLIP
Yifan Yang, Weiquan Huang, Yixuan Wei +8
Image token removal is an efficient augmentation strategy for reducing the cost of computing image features. However, this efficient augmentation strategy has been found to adverse…
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Zhihang Zhong, Mingxi Cheng, Zhirong Wu +7
Image cropping has progressed tremendously under the data-driven paradigm. However, current approaches do not account for the intentions of the user, which is an issue especially w…
Could Giant Pretrained Image Models Extract Universal Representations?
Yutong Lin, Ze Liu, Zheng Zhang +4
Frozen pretrained models have become a viable alternative to the pretraining-then-finetuning paradigm for transfer learning. However, with frozen models there are relatively few pa…
Training Overparametrized Neural Networks in Sublinear Time
Yichuan Deng, Hang Hu, Zhao Song +2
The success of deep learning comes at a tremendous computational and energy cost, and the scalability of training massively overparametrized neural networks is becoming a real barr…