2 citations · 4 across the 8 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
Jintao Zhang, Haofeng Huang, Pengle Zhang +3
Although quantization for linear layers has been widely used, its application to accelerate the attention process remains limited. To further enhance the efficiency of attention co…
cs.LG2024★ 2 cited
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
Jintao Zhang, Jia Wei, Haofeng Huang +3
The transformer architecture predominates across various models. As the heart of the transformer, attention has a computational complexity of , compared to for linea…
cs.CL2024★ 2 cited
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
Chaojun Xiao, Pengle Zhang, Xu Han +5
Large language models (LLMs) have emerged as a cornerstone in real-world applications with lengthy streaming inputs (e.g., LLM-driven agents). However, existing LLMs, pre-trained o…