On Design Principles for Efficient Heterogeneous DRAM-PIM-GPU Systems

Corey Lammie, Hadjer Benmeziane, William Andrew Simon +1

Heterogeneous DRAM-based processing-in-memory (PIM)-GPU systems promise significant efficiency gains for decode-phase large language model (LLM) inference, particularly in long-out…