LoMA: Lossless Compressed Memory Attention

Yumeng Wang, Zhenyang Xiao

Large Language Models (LLMs) face limitations due to the high demand on GPU memory and computational resources when handling long contexts. While sparsify the Key-Value (KV) cache…