Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters

Jinkyu Yim, Jaeyong Song, Yerim Choi +4

Training large language models (LLMs) is known to be challenging because of the huge computational and memory capacity requirements. To address these issues, it is common to use a…