most citedA2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder

1 citations · 1 across the 1 of their papers we have counts for

collaborators

1 paper

cs.CL2024★ 1 cited

A2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder

Hyun-rae Jo, Dongkun Shin

Recently, large language models (LLM) based on transformers are facing memory bottleneck issues due to KV cache, especially in long sequence handling. Previous researches proposed…