3 citations · 3 across the 2 of their papers we have counts for
2 papers
cs.LG2022
Wide Attention Is The Way Forward For Transformers?
Jason Ross Brown, Yiren Zhao, Ilia Shumailov +1
The Transformer is an extremely powerful and prominent deep learning architecture. In this work, we challenge the commonly held belief in deep learning that going deeper is better,…
cs.LG2022★ 3 cited
DARTFormer: Finding The Best Type Of Attention
Jason Ross Brown, Yiren Zhao, Ilia Shumailov +1
Given the wide and ever growing range of different efficient Transformer attention mechanisms, it is important to identify which attention is most effective when given a task. In t…