A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs

Hongsun Jang, Jaeyong Song, Changmin Shin +4

The computational and memory demands of large language models for generative inference present significant challenges for practical deployment. One promising solution targeting off…