consumer devices 1cpu-gpu offloading 1inference optimization 1large language models 1tensor scheduling 1
From the 1 of 3 linked papers with an AI index.
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices
Yangyijian Liu, Hongyi Ye, Mingyang Li +1
The paper introduces ATSInfer, a system that schedules tensor-level offloading between CPU and GPU to improve large language model inference on consumer devices, using static place…
cs.DC2025
PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices
Yangyijian Liu, Jun Li, Wu-Jun Li
The high memory and computation demand of large language models (LLMs) makes them challenging to be deployed on consumer devices due to limited GPU memory. Offloading can mitigate…