KVNAND: Efficient On-Device Large Language Model Inference Using DRAM-Free In-Flash Computing

Lishuo Deng, Shaojie Xu, Jinwu Chen +4

Deploying large language models (LLMs) on edge devices enables personalized agents with strong privacy and low cost. However, with tens to hundreds of billions of parameters, singl…