Joint Training on AMD and NVIDIA GPUs

Jon Hu, Thomas Jia, Jing Zhu +1

As large language models continue to scale, training demands on compute and system capacity grow rapidly, making single-vendor homogeneous clusters insufficient. This paper present…