4 papers
HyperParallel-FSDP: Topology-Aware Fully Sharded Training with Layout-Driven Muon on Ascend SuperPods
Mo Sun, Yifan Yao, Yanwei Liu +14
Declarative SPMD programming uses tensor sharding descriptions to drive distributed execution, separating parallelization from model code. However, the evaluated PyTorch DTensor st…
DisDP: Disaggregating Compute, Network, and Storage for Model-Sharded Data-Parallel Training
Mo Sun, Zihan Yang, Changyue Liao +5
Model-sharded data parallelism (MSDP), e.g., ZeRO, evenly shards the model states across all GPUs, and thus has been widely adopted by LLM pre-training, such as Llama and DeepSeek,…
DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
Jinwei Yao, Kaiqi Chen, Kexun Zhang +4
Large language models (LLMs) are increasingly employed for complex tasks that process multiple generation calls in a tree structure with shared prefixes of tokens, including few-sh…
LoHan: Low-Cost High-Performance Framework to Fine-Tune 100B Model on a Consumer GPU
Changyue Liao, Mo Sun, Zihan Yang +5
Nowadays, AI researchers become more and more interested in fine-tuning a pre-trained LLM, whose size has grown to up to over 100B parameters, for their downstream tasks. One appro…