429 citations · 2.5k across the 69 of their papers we have counts for
16 papers · 1 filter
Exploring Discrete Diffusion Models for Image Captioning
Zixin Zhu, Yixuan Wei, Jianfeng Wang +7
The image captioning task is typically realized by an auto-regressive method that decodes the text tokens one by one. We present a diffusion-based captioning model, dubbed the name…
Attentive Mask CLIP
Yifan Yang, Weiquan Huang, Yixuan Wei +8
Image token removal is an efficient augmentation strategy for reducing the cost of computing image features. However, this efficient augmentation strategy has been found to adverse…
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Zhihang Zhong, Mingxi Cheng, Zhirong Wu +7
Image cropping has progressed tremendously under the data-driven paradigm. However, current approaches do not account for the intentions of the user, which is an issue especially w…
Could Giant Pretrained Image Models Extract Universal Representations?
Yutong Lin, Ze Liu, Zheng Zhang +4
Frozen pretrained models have become a viable alternative to the pretraining-then-finetuning paradigm for transfer learning. However, with frozen models there are relatively few pa…
Training Overparametrized Neural Networks in Sublinear Time
Yichuan Deng, Hang Hu, Zhao Song +2
The success of deep learning comes at a tremendous computational and energy cost, and the scalability of training massively overparametrized neural networks is becoming a real barr…
Sublinear Time Algorithm for Online Weighted Bipartite Matching
Hang Hu, Zhao Song, Runzhou Tao +3
Online bipartite matching is a fundamental problem in online algorithms. The goal is to match two sets of vertices to maximize the sum of the edge weights, where for one set of ver…
Leveraging GAN Priors for Few-Shot Part Segmentation
Mengya Han, Heliang Zheng, Chaoyue Wang +3
Few-shot part segmentation aims to separate different parts of an object given only a few annotated samples. Due to the challenge of limited data, existing works mainly focus on le…
On Data Scaling in Masked Image Modeling
Zhenda Xie, Zheng Zhang, Yue Cao +4
An important goal of self-supervised learning is to enable model pre-training to benefit from almost unlimited data. However, one method that has recently become popular, namely ma…
Tutel: Adaptive Mixture-of-Experts at Scale
Changho Hwang, Wei Cui, Yifan Xiong +12
Sparsely-gated mixture-of-experts (MoE) has been widely adopted to scale deep learning models to trillion-plus parameters with fixed computational cost. The algorithmic performance…
Revealing the Dark Secrets of Masked Image Modeling
Zhenda Xie, Zigang Geng, Jingcheng Hu +3
Masked image modeling (MIM) as pre-training is shown to be effective for numerous vision downstream tasks, but how and where MIM works remain unclear. In this paper, we compare MIM…
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Hanpeng Hu, Chenyu Jiang, Yuchen Zhong +5
Distributed training using multiple devices (e.g., GPUs) has been widely adopted for learning DNN models over large datasets. However, the performance of large-scale distributed tr…
Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation
Yixuan Wei, Han Hu, Zhenda Xie +5
Masked image modeling (MIM) learns representations with remarkably good fine-tuning performances, overshadowing previous prevalent pre-training approaches such as image classificat…
iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition
Yixuan Wei, Yue Cao, Zheng Zhang +4
Image classification, which classifies images by pre-defined categories, has been the dominant approach to visual representation learning over the last decade. Visual learning thro…
Region Rebalance for Long-Tailed Semantic Segmentation
Jiequan Cui, Yuhui Yuan, Zhisheng Zhong +4
In this paper, we study the problem of class imbalance in semantic segmentation. We first investigate and identify the main challenges of addressing this issue through pixel rebala…
Adaptive Channel Allocation for Robust Differentiable Architecture Search
Chao Li, Jia Ning, Han Hu +1
Differentiable ARchiTecture Search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency. However, the excessive accumulation of the…
RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation
Haodi He, Yuhui Yuan, Xiangyu Yue +1
The segmentation task has traditionally been formulated as a complete-label pixel classification task to predict a class for each pixel from a fixed number of predefined semantic c…