6 papers
Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
Jungwoo Kim, Rubens Lacouture, Genghan Zhang +5
Mixture-of-Experts (MoE) has become a dominant architecture for scaling large language models (LLMs). However, the execution characteristics of MoE inference are changing rapidly a…
Regulating Branch Parallelism in LLM Serving
Swapnil Gandhi, Siva Hari, William J. Dally +1
Recent methods expose intra-request parallelism in LLM outputs, allowing independent branches to decode concurrently. Existing serving systems execute these branches eagerly or und…
Sparse Checkpointing for Fast and Reliable MoE Training
Swapnil Gandhi, Christos Kozyrakis
As large language models scale, training them requires thousands of GPUs over extended durations--making frequent failures an inevitable reality. While checkpointing remains the pr…
FailSafe: High-performance Resilient Serving
Ziyi Xu, Zhiqiang Xie, Swapnil Gandhi +1
Tensor parallelism (TP) enables large language models (LLMs) to scale inference efficiently across multiple GPUs, but its tight coupling makes systems fragile: a single GPU failure…
SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling
Athinagoras Skiadopoulos, Mark Zhao, Swapnil Gandhi +3
Mixture-of-Experts (MoE) models have become a widely-adopted solution to continue scaling model sizes without a corresponding linear increase in compute. During MoE model training,…
ReCycle: Resilient Training of Large DNNs using Pipeline Adaptation
Swapnil Gandhi, Mark Zhao, Athinagoras Skiadopoulos +1
Training large Deep Neural Network (DNN) models requires thousands of GPUs over the course of several days or weeks. At this scale, failures are frequent and can have a big impact…