Hardware-based Heterogeneous Memory Management for Large Language Model Inference

Soojin Hwang, Jungwoo Kim, Sanghyeon Lee +2

A large language model (LLM) is one of the most important emerging machine learning applications nowadays. However, due to its huge model size and runtime increase of the memory fo…