113 citations · 337 across the 24 of their papers we have counts for
4 papers · 1 filter
SimMIM: A Simple Framework for Masked Image Modeling
Zhenda Xie, Zheng Zhang, Yue Cao +5
This paper presents SimMIM, a simple framework for masked image modeling. We simplify recently proposed related approaches without special designs such as block-wise masking and to…
Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning
Shaobo Min, Qi Dai, Hongtao Xie +3
Cross-modal correlation provides an inherent supervision for video unsupervised representation learning. Existing methods focus on distinguishing different video clips by visual an…
On the Connection between Local Attention and Dynamic Depth-wise Convolution
Qi Han, Zejia Fan, Qi Dai +4
Vision Transformer (ViT) attains state-of-the-art performance in visual recognition, and the variant, Local Vision Transformer, makes further improvements. The major component in L…
Self-Supervised Learning with Swin Transformers
Zhenda Xie, Yutong Lin, Zhuliang Yao +4
We are witnessing a modeling shift from CNN to Transformers in computer vision. In this work, we present a self-supervised learning approach called MoBY, with Vision Transformers a…