2 citations · 5 across the 24 of their papers we have counts for
4 papers · 1 filter
ReMP: Low-Downtime Runtime Model-Parallelism Reconfiguration for LLM Serving
Haipeng Yuan, Kaining Zheng, Yongshu Bai +5
Current large language model (LLM) inference systems universally deploy ultra-large-scale models using a combination of Tensor Parallelism (TP) and Pipeline Parallelism (PP). Howev…
The Scalability for Parallel Machine Learning Training Algorithm: Dataset Matters
Daning Cheng, Hanping Zhang, Fen Xia +2
To gain a better performance, many researchers put more computing resource into an application. However, in the AI area, there is still a lack of a successful large-scale machine l…
Asynchronous COMID: the theoretic basis for transmitted data sparsification tricks on Parameter Server
Daning Cheng, Shigang Li, Yunquan Zhang
Asynchronous FTRL-proximal and L2 norm done at server are two widely used tricks in Parameters Server which is an implement of delayed SGD. Their commonness is leaving parts of upd…
The Research and Optimization of Parallel Finite Element Algorithm based on MiniFE
Meng Wu, Can Yang, Taoran Xiang +1
Finite element method (FEM) is one of the most important numerical methods in modern engineering design and analysis. Since traditional serial FEM is difficult to solve large FE pr…