81 citations · 414 across the 42 of their papers we have counts for
6 papers · 1 filter
Group Generalized Mean Pooling for Vision Transformer
Byungsoo Ko, Han-Gyu Kim, Byeongho Heo +4
Vision Transformer (ViT) extracts the final representation from either class token or an average of all patch tokens, following the architecture of Transformer in Natural Language…
Similarity of Neural Architectures using Adversarial Attack Transferability
Jaehui Hwang, Dongyoon Han, Byeongho Heo +3
In recent years, many deep neural architectures have been developed for image classification. Whether they are similar or dissimilar and what factors contribute to their (dis)simil…
Scratching Visual Transformer's Back with Uniform Attention
Nam Hyeon-Woo, Kim Yu-Ji, Byeongho Heo +3
The favorable performance of Vision Transformers (ViTs) is often attributed to the multi-head self-attention (MSA). The MSA enables global interactions at each layer of a ViT model…
Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation
Giung Nam, Hyungi Lee, Byeongho Heo +1
Ensembles of deep neural networks have demonstrated superior performance, but their heavy computational cost hinders applying them for resource-limited environments. It motivates d…
An Extendable, Efficient and Effective Transformer-based Object Detector
Hwanjun Song, Deqing Sun, Sanghyuk Chun +5
Transformers have been widely used in numerous vision problems especially for visual recognition and detection. Detection transformers are the first fully end-to-end learning syste…
Learning Features with Parameter-Free Layers
Dongyoon Han, YoungJoon Yoo, Beomyoung Kim +1
Trainable layers such as convolutional building blocks are the standard network design choices by learning parameters to capture the global context through successive spatial opera…