1 citations · 1 across the 5 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
Vikranth Srivatsa, Zijian He, Pu Guo +2
LLM serving is increasingly multi-tenant: the same deployment must handle latency-critical interactive requests and more relaxed background workloads under a fixed GPU budget. This…
cs.DC2024★ 1 cited
Preble: Efficient Distributed Prompt Scheduling for LLM Serving
Vikranth Srivatsa, Zijian He, Reyna Abhyankar +2
Prompts to large language models (LLMs) have evolved beyond simple user questions. For LLMs to solve complex problems, today's practices are to include domain-specific instructions…