works on
consumer devices 1cpu-gpu offloading 1inference optimization 1large language models 1tensor scheduling 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.DC2026

Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices

Yangyijian Liu, Hongyi Ye, Mingyang Li +1

The paper introduces ATSInfer, a system that schedules tensor-level offloading between CPU and GPU to improve large language model inference on consumer devices, using static place…