7 citations · 7 across the 9 of their papers we have counts for
4 papers · 1 filter
TokenButler: Token Importance is Predictable
Yash Akhauri, Ahmed F AbouElhamayed, Yifei Gao +4
Large Language Models (LLMs) rely on the Key-Value (KV) Cache to store token history, enabling efficient decoding of tokens. As the KV-Cache grows, it becomes a major memory and co…
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners
Yifei Gao, Jie Ou, Lei Wang +2
The quantization of large language models (LLMs) has been a prominent research area aimed at enabling their lightweight deployment in practice. Existing research about LLM's quanti…
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other
Yifei Gao, Jie Ou, Lei Wang +4
Emergent Large Language Models (LLMs) use their extraordinary performance and powerful deduction capacity to discern from traditional language models. However, the expenses of comp…
Pay Attention to What You Need
Yifei Gao, Shaohong Chen, Lei Wang +5
Although large language models (LLMs) have achieved significant success in natural language processing, they still struggle with long-context comprehension. Traditional approaches…