2 citations · 4 across the 13 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
FlashMask: Efficient and Rich Mask Extension of FlashAttention
Guoxia Wang, Jinle Zeng, Xiyuan Xiao +7
The computational and memory demands of vanilla attention scale quadratically with the sequence length , posing significant challenges for processing long sequences in Transform…
cs.CL2024★ 1 cited
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
Yilong Chen, Guoxia Wang, Junyuan Shang +7
Large Language Models (LLMs) have ignited an innovative surge of AI applications, marking a new era of exciting possibilities equipped with extended context windows. However, hosti…
cs.CR2024★ 1 cited
A Framework for Cost-Effective and Self-Adaptive LLM Shaking and Recovery Mechanism
Zhiyu Chen, Yu Li, Suochao Zhang +4
As Large Language Models (LLMs) gain great success in real-world applications, an increasing number of users are seeking to develop and deploy their customized LLMs through cloud s…