most citedUnderstanding Gaussian Attention Bias of Vision Transformers Using Effective Receptive Fields

7 citations · 12 across the 6 of their papers we have counts for

collaborators

6 papers

cs.CV2024

Scale Equalization for Multi-Level Feature Fusion

Bum Jun Kim, Sang Woo Kim

Deep neural networks have exhibited remarkable performance in a variety of computer vision fields, especially in semantic segmentation tasks. Their success is often attributed to m…

cs.CV2023

Analysis of NaN Divergence in Training Monocular Depth Estimation Model

Bum Jun Kim, Hyeonah Jang, Sang Woo Kim

The latest advances in deep learning have facilitated the development of highly accurate monocular depth estimation models. However, when training a monocular depth estimation netw…

cs.CV20233 cited

Resolution-Aware Design of Atrous Rates for Semantic Segmentation Networks

Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang +1

DeepLab is a widely used deep neural network for semantic segmentation, whose success is attributed to its parallel architecture called atrous spatial pyramid pooling (ASPP). ASPP…

cs.CV20237 cited

Understanding Gaussian Attention Bias of Vision Transformers Using Effective Receptive Fields

Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang +1

Vision transformers (ViTs) that model an image as a sequence of partitioned patches have shown notable performance in diverse vision tasks. Because partitioning patches eliminates…

cs.LG20232 cited

How to Use Dropout Correctly on Residual Networks with Batch Normalization

Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang +2

For the stable optimization of deep neural networks, regularization methods such as dropout and batch normalization have been used in various tasks. Nevertheless, the correct posit…

cs.LG2023

On the Ideal Number of Groups for Isometric Gradient Propagation

Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang +1

Recently, various normalization layers have been proposed to stabilize the training of deep neural networks. Among them, group normalization is a generalization of layer normalizat…