452 citations · 469 across the 7 of their papers we have counts for
4 papers · 1 filter
Wavelet-Based Image Tokenizer for Vision Transformers
Zhenhai Zhu, Radu Soricut
Non-overlapping patch-wise convolution is the default image tokenizer for all state-of-the-art vision Transformer (ViT) models. Even though many ViT variants have been proposed to…
Continual Learning in Open-vocabulary Classification with Complementary Memory Systems
Zhen Zhu, Weijie Lyu, Yao Xiao +1
We introduce a method for flexible and efficient continual learning in open-vocabulary image classification, drawing inspiration from the complementary learning systems observed in…
Multimodal Pretraining for Dense Video Captioning
Gabriel Huang, Bo Pang, Zhenhai Zhu +2
Learning specific hands-on skills such as cooking, car maintenance, and home repairs increasingly happens via instructional videos. The user experience with such videos is known to…
Improved Image Captioning via Policy Gradient optimization of SPIDEr
Siqi Liu, Zhenhai Zhu, Ning Ye +2
Current image captioning methods are usually trained via (penalized) maximum likelihood estimation. However, the log-likelihood score of a caption does not correlate well with huma…