196 citations · 543 across the 22 of their papers we have counts for
Showing 2020 · cs.CLShow all
2 papers · 2 filters
cs.CL2020★ 5 cited
HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
Yi Tay, Zhe Zhao, Dara Bahri +2
Achieving state-of-the-art performance on natural language understanding tasks typically relies on fine-tuning a fresh model for every task. Consequently, this approach leads to a…
cs.CL2020★ 196 cited
Synthesizer: Rethinking Self-Attention in Transformer Models
Yi Tay, Dara Bahri, Donald Metzler +3
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models. But is it really required? This paper investigates the true importan…