7 citations · 13 across the 8 of their papers we have counts for
4 papers · 1 filter
TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Kan Zhu, Mathew Jacob, Chenxi Ma +4
Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real wor…
M*: A Modular, Extensible, Serving System for Multimodal Models
Atindra Jha, Naomi Sagan, Keisuke Kamahori +9
We are entering a new era of composite model architectures that integrate diverse components such as vision encoders, language backbones, diffusion and flow heads, audio codecs, ac…
DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling
Yi Pan, Yile Gu, Jinbin Luo +7
Intra-device parallelism addresses resource under-utilization in ML inference and training by overlapping the execution of operators with different resource usage. However, its wid…
VoxServe: Streaming-Centric Serving System for Speech Language Models
Keisuke Kamahori, Wei-Tzu Lee, Atindra Jha +4
Deploying modern Speech Language Models (SpeechLMs) in streaming settings requires systems that provide low latency, high throughput, and strong guarantees of streamability. Existi…