Lumos: Efficient Performance Modeling and Estimation for Large-scale LLM Training

Mingyu Liang, Hiwot Tadese Kassa, Wenyin Fu +3

Training LLMs in distributed environments presents significant challenges due to the complexity of model execution, deployment systems, and the vast space of configurable strategie…