HyperParallel-Mpipe: A Composable Algebra System for Optimizing MLLM Training over Supernode Clusters

Chong Li, Zhengdao Yu, Nelson Lossing +6

Modern AI applications have expanded beyond text-only interaction into a wide range of multimodal scenarios, making multimodal large language models (MLLMs) crucial for both resear…