17 citations · 30 across the 7 of their papers we have counts for
4 papers · 1 filter
Leveraging redundancy in attention with Reuse Transformers
Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit +5
Pairwise dot product-based attention allows Transformers to exchange information between tokens in an input-dependent way, and is key to their success across diverse applications i…
Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation
Srinadh Bhojanapalli, Ayan Chakrabarti, Himanshu Jain +3
State-of-the-art transformer models use pairwise dot-product based self-attention, which comes at a computational cost quadratic in the input sequence length. In this paper, we inv…
Understanding Robustness of Transformers for Image Classification
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner +3
Deep Convolutional Neural Networks (CNNs) have long been the architecture of choice for computer vision tasks. Recently, Transformer-based architectures like Vision Transformer (Vi…
On the Reproducibility of Neural Network Predictions
Srinadh Bhojanapalli, Kimberly Wilber, Andreas Veit +4
Standard training techniques for neural networks involve multiple sources of randomness, e.g., initialization, mini-batch ordering and in some cases data augmentation. Given that n…