7 citations · 18 across the 10 of their papers we have counts for
5 papers · 1 filter
Temperature-Free Loss Function for Contrastive Learning
Bum Jun Kim, Sang Woo Kim
As one of the most promising methods in self-supervised learning, contrastive learning has achieved a series of breakthroughs across numerous fields. A predominant approach to impl…
Stochastic Subsampling With Average Pooling
Bum Jun Kim, Sang Woo Kim
Regularization of deep neural networks has been an important issue to achieve higher generalization performance without overfitting problems. Although the popular method of Dropout…
The Disappearance of Timestep Embedding in Modern Time-Dependent Neural Networks
Bum Jun Kim, Yoshinobu Kawahara, Sang Woo Kim
Dynamical systems are often time-varying, whose modeling requires a function that evolves with respect to time. Recent studies such as the neural ordinary differential equation pro…
How to Use Dropout Correctly on Residual Networks with Batch Normalization
Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang +2
For the stable optimization of deep neural networks, regularization methods such as dropout and batch normalization have been used in various tasks. Nevertheless, the correct posit…
On the Ideal Number of Groups for Isometric Gradient Propagation
Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang +1
Recently, various normalization layers have been proposed to stabilize the training of deep neural networks. Among them, group normalization is a generalization of layer normalizat…