21 citations · 25 across the 5 of their papers we have counts for
5 papers
ReMP: Low-Downtime Runtime Model-Parallelism Reconfiguration for LLM Serving
Haipeng Yuan, Kaining Zheng, Yongshu Bai +5
Current large language model (LLM) inference systems universally deploy ultra-large-scale models using a combination of Tensor Parallelism (TP) and Pipeline Parallelism (PP). Howev…
BitSnap: Checkpoint Sparsification and Quantization in LLM Training
Yanxin Peng, Qingping Li, Baodong Wu +4
As large language models (LLMs) continue to grow in size and complexity, efficient checkpoint saving\&loading has become crucial for managing storage, memory usage, and fault toler…
TRANSOM: An Efficient Fault-Tolerant System for Training LLMs
Baodong Wu, Lei Xia, Qingping Li +6
Large language models (LLMs) with hundreds of billions or trillions of parameters, represented by chatGPT, have achieved profound impact on various fields. However, training LLMs w…
AGCM-3DLF: Accelerating Atmospheric General Circulation Model via 3D Parallelization and Leap-Format
Hang Cao, Liang Yuan, He Zhang +1
The Atmospheric General Circulation Model (AGCM) has been an important research tool in the study of climate change for decades. As the demand for high-resolution simulation is bec…
The dynamic parallel distribution algorithm for hybrid density-functional calculations in HONPAS package
Honghui Shang, Lei Xu, Baodong Wu +3
This work presents a dynamic parallel distribution scheme for the Hartree-Fock exchange~(HFX) calculations based on the real-space NAO2GTO framework. The most time-consuming electr…