most citedDesigning BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling

37 citations · 37 across the 1 of their papers we have counts for

collaborators
Showing cs.CVShow all

1 paper · 1 filter

cs.CV2023★ 37 cited

Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling

Keyu Tian, Yi Jiang, Qishuai Diao +3

We identify and overcome two key obstacles in extending the success of BERT-style pre-training, or the masked image modeling, to convolutional networks (convnets): (i) convolution…