1 citations · 1 across the 4 of their papers we have counts for
3 papers · 1 filter
CIERA: Cross-Iteration Exponent Reuse for Lossless Allgather in Sharded MoE Training
Ali Zafar Sadiq, Haiying Shen, Masahiro Tanaka
In training Mixture-of-Experts (MoE) models, sharded data parallelism partitions each expert's parameters across GPUs, requiring an Allgather operation to reconstruct the full weig…
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
Bing Xie, Zhipeng Wang, Masahiro Tanaka +2
We present PEEK, a lightweight scheduling and eviction framework for both online (streaming) and offline (batch) LLM serving; this paper focuses on the online regime. PEEK maintain…
DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
Masahiro Tanaka, Du Li, Umesh Chand +3
The rapid growth of deep learning models has increased the demand for efficient distributed training strategies. Fully sharded approaches like ZeRO-3 and FSDP partition model param…