1 citations · 1 across the 7 of their papers we have counts for
3 papers · 1 filter
Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
Vikranth Srivatsa, Zijian He, Pu Guo +2
LLM serving is increasingly multi-tenant: the same deployment must handle latency-critical interactive requests and more relaxed background workloads under a fixed GPU budget. This…
VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU
Zijian He, Adrian Sampson, Yiying Zhang +1
Modern GPUs increasingly rely on specialized and asynchronous hardware units to deliver high performance. Yet these units are often underutilized because today's GPU software stack…
Preble: Efficient Distributed Prompt Scheduling for LLM Serving
Vikranth Srivatsa, Zijian He, Reyna Abhyankar +2
Prompts to large language models (LLMs) have evolved beyond simple user questions. For LLMs to solve complex problems, today's practices are to include domain-specific instructions…