9 citations · 26 across the 20 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CV2021
Ripple Attention for Visual Perception with Sub-quadratic Complexity
Lin Zheng, Huijie Pan, Lingpeng Kong
Transformer architectures are now central to sequence modeling tasks. At its heart is the attention mechanism, which enables effective modeling of long-term dependencies in a seque…
cs.CL2021★ 1 cited
Cascaded Head-colliding Attention
Lin Zheng, Zhiyong Wu, Lingpeng Kong
Transformers have advanced the field of natural language processing (NLP) on a variety of important tasks. At the cornerstone of the Transformer architecture is the multi-head atte…