32 citations · 79 across the 17 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 9 cited
HyperAttention: Long-context Attention in Near-Linear Time
Insu Han, Rajesh Jayaram, Amin Karbasi +3
We present an approximate attention mechanism named HyperAttention to address the computational challenges posed by the growing complexity of long contexts used in Large Language M…
cs.LG2023★ 5 cited
KDEformer: Accelerating Transformers via Kernel Density Estimation
Amir Zandieh, Insu Han, Majid Daliri +1
Dot-product attention mechanism plays a crucial role in modern deep architectures (e.g., Transformer) for sequence modeling, however, naïve exact computation of this model incurs q…