ShardMeter: Sharded and Geo-Distributed Training Without the Guesswork

Tim Beringer, Patrick Diem, Felix Wolf +1

Training large-scale AI models often outgrows a single data center, demanding sharded, multi-cluster, and decentralized training. However, the huge space of resource allocations ma…