BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training

Zili Zhang, Chengxu Yang, Shenglong Zhang +8

Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. Existing systems redesign the training pipeline to address these challenges, b…