16 citations · 33 across the 5 of their papers we have counts for
3 papers · 1 filter
Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines
Marcel Wagenländer, Otto White, Britannio Jarrett +9
Agentic workflows carry out complex tasks by orchestrating multiple large language models (LLMs) and tools. Serving them at a target throughput with low latency is hard because the…
Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections
Marcel Wagenländer, Guo Li, Bo Zhao +2
Deep learning (DL) jobs use multi-dimensional parallelism, i.e. combining data, model, and pipeline parallelism, to use large GPU clusters efficiently. Long-running jobs may experi…
Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness
Zeyuan Tan, Xiulong Yuan, Congjie He +7
Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number o…