Bandwidth-Aware LLM Inference on Heterogeneous Many-Core Supercomputers

Yao Lu, Zhongzhi Luan, Gen Li +6

Large language model (LLM) inference is limited by high computational cost and memory bandwidth demands, making deployment on heterogeneous many-core processors challenging. Taking…