Galvatron: An Automatic Distributed System for Efficient Foundation Model Training

Xinyi Liu, Yujie Wang, Shenhan Zhu +4

Galvatron is a distributed system for efficiently training large-scale Foundation Models. It overcomes the complexities of selecting optimal parallelism strategies by automatically…