Boosting Distributed Training Performance of the Unpadded BERT Model

Jinle Zeng, Min Li, Zhihua Wu +4

Pre-training models are an important tool in Natural Language Processing (NLP), while the BERT model is a classic pre-training model whose structure has been widely adopted by foll…