ROMA: a Read-Only-Memory-based Accelerator for QLoRA-based On-Device LLM

Wenqiang Wang, Yijia Zhang, Zikai Zhang +4

As large language models (LLMs) demonstrate powerful capabilities, deploying them on edge devices has become increasingly crucial, offering advantages in privacy and real-time inte…