121 citations · 199 across the 10 of their papers we have counts for
3 papers · 1 filter
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers
Michael Hassid, Hao Peng, Daniel Rotem +4
The attention mechanism is considered the backbone of the widely-used Transformer architecture. It contextualizes the input by computing input-specific attention matrices. We find…
Modeling Context With Linear Attention for Scalable Document-Level Translation
Zhaofeng Wu, Hao Peng, Nikolaos Pappas +1
Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations. However, these models, predominantly based on transfo…
XLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction
Yuwei Cao, William Groves, Tanay Kumar Saha +4
Temporal Expression Extraction (TEE) is essential for understanding time in natural language. It has applications in Natural Language Processing (NLP) tasks such as question answer…