Ecco: Improving Memory Bandwidth and Capacity for LLMs via Entropy-aware Cache Compression

Feng Cheng, Cong Guo, Chiyue Wei +7

Large language models (LLMs) have demonstrated transformative capabilities across diverse artificial intelligence applications, yet their deployment is hindered by substantial memo…