2 papers
cs.CV2024
Improving Generative Pre-Training: An In-depth Study of Masked Image Modeling and Denoising Models
Hyesong Choi, Daeun Kim, Sungmin Cha +2
In this work, we dive deep into the impact of additive noise in pre-training deep networks. While various methods have attempted to use additive noise inspired by the success of la…
cs.CV2024
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training
Hyesong Choi, Hyejin Park, Kwang Moo Yi +2
In this paper, we introduce Saliency-Based Adaptive Masking (SBAM), a novel and cost-effective approach that significantly enhances the pre-training performance of Masked Image Mod…