8 papers
ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving
Seokjin Go, Marko Scrbak, Ephrem Wu +2
In distributed Mixture-of-Experts (MoE) inference, input-dependent token routing interacts with GPU performance variability to create persistent stragglers under synchronized execu…
NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning
Irene Wang, Vishnu Varma Venkata, Arvind Krishnamurthy +1
The growing scale of deep learning demands distributed training frameworks that jointly reason about parallelism, memory, and network topology. Prior works often rely on heuristic…
Stream2LLM: Overlap Context Streaming and Prefill for Reduced Time-to-First-Token (TTFT)
Rajveer Bachkaniwala, Chengqi Luo, Richard So +2
Context retrieval systems for LLM inference face a critical challenge: high retrieval latency creates a fundamental tension between waiting for complete context (poor time-to-first…
Equinox: Decentralized Scheduling for Hardware-Aware Orbital Intelligence
Ansel Kaplan Erol, Divya Mahajan
Earth-observation satellites are emerging as distributed edge platforms for time-critical tasks, yet orbital scheduling remains challenged by intermittent energy harvesting and tem…
VectorLiteRAG: Latency-Aware and Fine-Grained Resource Partitioning for Efficient RAG
Junkyum Kim, Divya Mahajan
Retrieval-Augmented Generation (RAG) systems combine vector similarity search with large language models (LLMs) to deliver accurate, context-aware responses. However, co-locating t…
Characterizing the Efficiency of Distributed Training: A Power, Performance, and Thermal Perspective
Seokjin Go, Joongun Park, Spandan More +5
The rapid scaling of Large Language Models (LLMs) has pushed training workloads far beyond the limits of single-node analysis, demanding a deeper understanding of how these models…