1 paper
Junxiong Wang, Jing Nathan Yan, Albert Gu +1
Transformers have been essential to pretraining success in NLP. While other architectures have been used, downstream accuracy is either significantly worse, or requires attention l…