HPIM: Heterogeneous Processing-In-Memory-based Accelerator for Large Language Models Inference

Cenlin Duan, Jianlei Yang, Rubing Yang +8

The deployment of large language models (LLMs) presents significant challenges due to their enormous memory footprints, low arithmetic intensity, and stringent latency requirements…