4 papers
PLoRA: An NDP-Enhanced Pooled-Memory System for Cost-Efficient Multi-LoRA Serving
Zhongkai Yu, Ohm Rishabh Venkatachalam, Zheng Wang +9
Multi-LoRA serving is how one base model becomes thousands of specialized variants, one adapter per user, task, or agent, and the deployments can hold 1000-plus adapters. Serving t…
FlashCP: Load-Balanced Communication-Efficient Context Parallelism for LLM Training
Zheng Wang, Eric Liu, Linan Jiang +5
Context parallelism (CP) is essential for training large-scale, long-context language models, as it partitions sequences to reduce memory overhead. However, existing CP methods suf…
HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
Zhengding Hu, Vibha Murthy, Zaifeng Pan +4
This paper addresses emerging system-level challenges in heterogeneous retrieval-augmented generation (RAG) serving, where complex multi-stage workflows and diverse request pattern…
Faith: An Efficient Framework for Transformer Verification on GPUs
Boyuan Feng, Tianqi Tang, Yuke Wang +5
Transformer verification draws increasing attention in machine learning research and industry. It formally verifies the robustness of transformers against adversarial attacks such…