1 citations · 1 across the 4 of their papers we have counts for
13 papers
TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Kan Zhu, Mathew Jacob, Chenxi Ma +4
Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real wor…
Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended
Aditya K Kamath, Arvind Krishnamurthy, Marco Canini +1
Machine learning (ML) training and inference often process data sets far exceeding GPU memory capacity, forcing them to rely on PCIe for on-demand tensor transfers, causing critica…
NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning
Irene Wang, Vishnu Varma Venkata, Arvind Krishnamurthy +1
The growing scale of deep learning demands distributed training frameworks that jointly reason about parallelism, memory, and network topology. Prior works often rely on heuristic…
TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval
Chien-Yu Lin, Keisuke Kamahori, Yiyu Liu +11
Retrieval-augmented generation (RAG) extends large language models (LLMs) with external data sources to enhance factual correctness and domain coverage. Modern RAG pipelines rely o…
GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads
Fanjiang Ye, Zhangke Li, Xinrui Zhong +10
Diffusion models have emerged as the prevailing approach for text-to-image (T2I) and text-to-video (T2V) generation, yet production platforms must increasingly serve both modalitie…
FAST: An Efficient Scheduler for All-to-All GPU Communication
Yiran Lei, Dongjoo Lee, Liangyu Zhao +9
All-to-All(v) communication is a critical primitive in modern machine learning workloads, particularly mixture-of-experts (MoE) models. Unfortunately, efficient scheduling is chall…