81 citations · 414 across the 42 of their papers we have counts for
6 papers · 1 filter
SeiT++: Masked Token Modeling Improves Storage-efficient Training
Minhyun Lee, Song Park, Byeongho Heo +2
Recent advancements in Deep Neural Network (DNN) models have significantly improved performance across computer vision tasks. However, achieving highly generalizable and high-perfo…
Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Images
Jiwon Kim, Byeongho Heo, Sangdoo Yun +2
Semantic correspondence methods have advanced to obtaining high-quality correspondences employing complicated networks, aiming to maximize the model capacity. However, despite the…
Learning with Unmasked Tokens Drives Stronger Vision Learners
Taekyung Kim, Sanghyuk Chun, Byeongho Heo +1
Masked image modeling (MIM) has become a leading self-supervised learning strategy. MIMs such as Masked Autoencoder (MAE) learn strong representations by randomly masking input tok…
Masking meets Supervision: A Strong Learning Alliance
Byeongho Heo, Taekyung Kim, Sangdoo Yun +1
Pre-training with random masked inputs has emerged as a novel trend in self-supervised training. However, supervised learning still faces a challenge in adopting masking augmentati…
What Do Self-Supervised Vision Transformers Learn?
Namuk Park, Wonjae Kim, Byeongho Heo +2
We present a comparative study on how and why contrastive learning (CL) and masked image modeling (MIM) differ in their representations and in their performance of downstream tasks…
SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage
Song Park, Sanghyuk Chun, Byeongho Heo +2
We need billion-scale images to achieve more generalizable and ground-breaking vision models, as well as massive dataset storage to ship the images (e.g., the LAION-4B dataset need…