4 papers
Temperature-Free Loss Function for Contrastive Learning
Bum Jun Kim, Sang Woo Kim
As one of the most promising methods in self-supervised learning, contrastive learning has achieved a series of breakthroughs across numerous fields. A predominant approach to impl…
Stochastic Subsampling With Average Pooling
Bum Jun Kim, Sang Woo Kim
Regularization of deep neural networks has been an important issue to achieve higher generalization performance without overfitting problems. Although the popular method of Dropout…
The Disappearance of Timestep Embedding in Modern Time-Dependent Neural Networks
Bum Jun Kim, Yoshinobu Kawahara, Sang Woo Kim
Dynamical systems are often time-varying, whose modeling requires a function that evolves with respect to time. Recent studies such as the neural ordinary differential equation pro…
Configuring Data Augmentations to Reduce Variance Shift in Positional Embedding of Vision Transformers
Bum Jun Kim, Sang Woo Kim
Vision transformers (ViTs) have demonstrated remarkable performance in a variety of vision tasks. Despite their promising capabilities, training a ViT requires a large amount of di…