300 citations · 1.5k across the 38 of their papers we have counts for
Showing 2018Show all
3 papers · 1 filter
cs.CL2018★ 5 cited
Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together
Tao Shen, Tianyi Zhou, Guodong Long +2
Neural networks equipped with self-attention have parallelizable computation, light-weight structure, and the ability to capture both long-range and local dependencies. Further, th…
cs.CL2018★ 77 cited
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
Tao Shen, Tianyi Zhou, Guodong Long +2
Recurrent neural networks (RNN), convolutional neural networks (CNN) and self-attention networks (SAN) are commonly used to produce context-aware representations. RNN can capture l…
cs.CL2018★ 26 cited
Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
Tao Shen, Tianyi Zhou, Guodong Long +3
Many natural language processing tasks solely rely on sparse dependencies between a few tokens in a sentence. Soft attention mechanisms show promising performance in modeling local…