195 citations · 936 across the 38 of their papers we have counts for
7 papers · 1 filter
Efficient AlphaFold2 Training using Parallel Evoformer and Branch Parallelism
Guoxia Wang, Zhihua Wu, Xiaomin Fang +4
The accuracy of AlphaFold2, a frontier end-to-end structure prediction system, is already close to that of the experimental determination techniques. Due to the complex model archi…
Boosting Distributed Training Performance of the Unpadded BERT Model
Jinle Zeng, Min Li, Zhihua Wu +4
Pre-training models are an important tool in Natural Language Processing (NLP), while the BERT model is a classic pre-training model whose structure has been widely adopted by foll…
Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources
Ji Liu, Daxiang Dong, Xi Wang +5
Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which ex…
HelixFold: An Efficient Implementation of AlphaFold2 using PaddlePaddle
Guoxia Wang, Xiaomin Fang, Zhihua Wu +6
Accurate protein structure prediction can significantly accelerate the development of life science. The accuracy of AlphaFold2, a frontier end-to-end structure prediction system, i…
MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
Dianhai Yu, Liang Shen, Hongxiang Hao +5
While modern internet services, such as chatbots, search engines, and online advertising, demand the use of large-scale deep neural networks (DNNs), distributed training and infere…
End-to-end Adaptive Distributed Training on PaddlePaddle
Yulong Ao, Zhihua Wu, Dianhai Yu +10
Distributed training has become a pervasive and effective approach for training a large neural network (NN) model with processing massive data. However, it is very challenging to s…
HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments
Ji Liu, Zhihua Wu, Dianhai Yu +6
Deep neural networks (DNNs) exploit many layers and a large number of parameters to achieve excellent performance. The training process of DNN models generally handles large-scale…