5 citations · 5 across the 8 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.AI2025
SlimPack: Fine-Grained Asymmetric Packing for Balanced and Efficient Variable-Length LLM Training
Yuliang Liu, Guohao Wu, Shenglong Zhang +4
The efficient distributed training of Large Language Models (LLMs) is severely hampered by the extreme variance in context lengths. This data heterogeneity, amplified by convention…
cs.LG2025
SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
Zhouyang Li, Yuliang Liu, Wei Zhang +4
Pipeline Parallelism (PP) serves as a crucial technique for training Large Language Models (LLMs), owing to its capability to alleviate memory pressure from model states with relat…