2 papers
cs.DC2026
Layer-wise CPU-GPU Scheduling for LLM Inference on Memory-Limited Consumer GPUs
Naoto Sugiura, Yuta Tokusashi, Hiroki Matsutani
As large language model inference environments expand toward consumer devices, it remains challenging to efficiently run models that exceed GPU memory capacity in single-GPU settin…
cs.LG2026
Accelerating Local LLMs on Resource-Constrained Edge Devices via Distributed Prompt Caching
Hiroki Matsutani, Naoki Matsuda, Naoto Sugiura
Since local LLM inference on resource-constrained edge devices imposes a severe performance bottleneck, this paper proposes distributed prompt caching to enhance inference performa…