168 citations · 503 across the 25 of their papers we have counts for
3 papers · 1 filter
Leveraging redundancy in attention with Reuse Transformers
Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit +5
Pairwise dot product-based attention allows Transformers to exchange information between tokens in an input-dependent way, and is key to their success across diverse applications i…
EncT5: A Framework for Fine-tuning T5 as Non-autoregressive Models
Frederick Liu, Terry Huang, Shihang Lyu +3
Pre-trained encoder-decoder transformer architectures have become increasingly popular recently with the advent of T5 models. T5 has also become more favorable over other architect…
Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
Jiefeng Chen, Frederick Liu, Besim Avci +3
When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance…