most citedPatrickStar: Parallel Training of Pre-trained Models via Chunk-based Memory Management

35 citations · 35 across the 1 of their papers we have counts for

collaborators

1 paper

cs.LG2021★ 35 cited

PatrickStar: Parallel Training of Pre-trained Models via Chunk-based Memory Management

Jiarui Fang, Zilin Zhu, Shenggui Li +4

The pre-trained model (PTM) is revolutionizing Artificial Intelligence (AI) technology. However, the hardware requirement of PTM training is prohibitively high, making it a game fo…