2 papers
cs.DC2026
Efficient Training on Multiple Consumer GPUs with RoundPipe
Yibin Luo, Shiwei Gao, Huichuan Zheng +2
Fine-tuning Large Language Models (LLMs) on consumer-grade GPUs is highly cost-effective, yet constrained by limited GPU memory and slow PCIe interconnects. Pipeline parallelism co…
cs.DC2024
Fast State Restoration in LLM Serving with HCache
Shiwei Gao, Youmin Chen, Jiwu Shu
The growing complexity of LLM usage today, e.g., multi-round conversation and retrieval-augmented generation (RAG), makes contextual states (i.e., KV cache) reusable across user re…