consumer devices 1cpu-gpu offloading 1inference optimization 1large language models 1tensor scheduling 1
From the 1 of 3 linked papers with an AI index.
3 papers
cs.DC2026
Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices
Yangyijian Liu, Hongyi Ye, Mingyang Li +1
The paper introduces ATSInfer, a system that schedules tensor-level offloading between CPU and GPU to improve large language model inference on consumer devices, using static place…
cs.DC2025
PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices
Yangyijian Liu, Jun Li, Wu-Jun Li
The high memory and computation demand of large language models (LLMs) makes them challenging to be deployed on consumer devices due to limited GPU memory. Offloading can mitigate…
cs.LG2025
LCQ: Low-Rank Codebook based Quantization for Large Language Models
Wen-Pu Cai, Ming-Yang Li, Wu-Jun Li
Large language models~(LLMs) have recently demonstrated promising performance in many tasks. However, the high storage and computational cost of LLMs has become a challenge for dep…